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This  document  reports  the  work  performed  by  the  Institute  for  Defense  Analyses 
for  the  Director,  Acquisition  Resources  and  Analysis,  Office  of  the  Under  Secretary  of 
Defense  (Acquisition,  Technology,  and  Logistics)  in  fulfillment  of  the  task  entitled 
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EXECUTIVE  SUMMARY 


The  Institute  for  Defense  Analyses  (IDA),  assisted  by  the  CNA  Corporation,  was 
tasked  by  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  [OUSD(AT&L)]  to  examine  recent  acquisition  refonn  initiatives  and  to  make 
appropriate  recommendations  for  further  efforts  by  the  Undersecretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  [USD(AT&L)j.  Our  review  focused  on  the 
processes  and  procedures  for  the  long-term  acquisition  of  military  equipment  and  systems 
essential  for  properly  equipping  and  supporting  the  future  military  force  structure.  This 
report  excludes  (1)  ongoing  initiatives  to  improve  the  procedures  for  rapidly  responding 
to  emerging  needs  of  current  operations,  particularly  in  Iraq,  and  (2)  a  broad  treatment  of 
excessive  acquisition  cost  growth — an  important  topic  that  is  the  subject  of  a  separate 
ongoing  IDA  study. 

In  this  Phase  II  report  the  team  proposes  three  major  sets  of  initiatives  (discussed 
in  the  next  three  sections)  and  one  minor  initiative  (discussed  last)  for  further  action  by 
the  USD(AT&L). 

I.  IMPROVED  PROGRAM  STABILITY  AND  “JOINTNESS”  THROUGH 
BETTER  INTEGRATION  OF  REQUIREMENTS/PROGRAMMING/ 
ACQUISITION  PROCESSES 

Most  of  the  Phase  I  Report  initiatives  recommended  for  further  consideration  can 
be  characterized  as  “good  ideas  that  were  never  fully  implemented.”  Chief  among  them 
is  the  Packard  Commission  recommendation  for  better  integrating  the  Defense 
Department’s  acquisition,  requirements,  and  program/budget  processes,  with  a  particular 
emphasis  on  clarifying  and  strengthening  the  role  of  the  Secretary’s  office  in  deciding 
what  to  buy  (the  requirements  process),  not  just  how  to  buy  it.  This  is  particularly 
important  for  programs  that  are  essential  to  effective  and  efficient  joint  military 
operations,  such  as  command  and  control  systems.  Closely  related  is  the  need  for  the 
acquisition  executives  to  work  within  the  resource  allocation  process  to  improve 
acquisition  program  funding  stability.  A  specific  improvement  would  be  to  push  for  more 
accurate  programming  and  budgeting  for  the  major  non-acquisition  elements  of  the 
DoD  program — mainly  military  personnel  and  operations/maintenance  costs.  This  broad 
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initiative  encompasses  requirements  formulation,  acquisition  management,  and  joint 
programming,  as  detailed  below. 

A.  Requirements  Formulation 

Under  the  law  (though  never  strongly  invoked),  the  USD(AT&L)  already  is 
responsible  for  “requirements”  because  he  is  empowered  to  decide  what  weapons 
systems  (including  key  characteristics)  DoD  will  propose  to  buy.1  The  Joint  Staffs  Joint 
Requirement  Oversight  Council/Joint  Capabilities  Integration  and  Development  System 
(JROC/JCIDS)  process  provides  the  fiscally  unconstrained  requirements  advice  of  the 
warfighters,  but  not  the  requirements  decision.  Such  advice  is  routinely  provided  not 
only  by  the  Chainnan  of  the  Joint  Chiefs  of  Staff,  but  also  by  the  Service  Chiefs  and, 
increasingly,  by  the  Combatant  Commanders.  As  a  practical  matter,  the  Service  Chiefs 
are  the  primary  source  of  proposals  for  new  weapons  systems  and  decisions  to  change 
them. 

The  key  vehicle  for  integrating  perceived  needs  for  new  weapon  system 
development  into  the  larger  DoD  resource  management  process  is  the  OSD-managed 
broad  Analysis  of  Alternatives  (AoA).  Current  directives  call  for  an  AoA  prior  to  the 
convening  of  the  Milestone  A  review  by  the  Defense  Acquisition  Board  (DAB)  that  sets 
the  stage  for  a  potential  new  acquisition.2  In  recent  years,  however,  early  AoAs  have 
been  neglected,  in  part  perhaps  because  directives  have  defined  fonnal  acquisition 
program  initiation  to  occur  at  Milestone  B.  Experience  has  shown  that  if  the  Defense 
Acquisition  Executive’s  (DAE)  attention  to  potential  large  new  acquisition  programs  is 
delayed  until  Milestone  B,  it  can  be  inordinately  difficult  for  him  to  rectify  earlier 
missteps. 

Recommendation : 

The  Under  Secretary  of  Defense  (AT&L)  should  lead  or  co-lead  a  formal  AoA 
that  (1)  identifies  and  evaluates  a  broad  range  of  potential  solutions  to  a  perceived  need 
for  new  military  capabilities  involving  new  weapons  systems,  and  (2)  occurs  well  before 
a  Milestone  A  commitment  is  made  to  pursue  any  particular  set  of  material  solutions. 


1  In  this  document  “requirements”  refers  to  the  realistic,  achievable,  and  affordable  statements  of  the 
intended  principal  characteristics  of  a  notional  new  weapon  system. 

2  DoDI  5000.2,  May  12  2003. 
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B.  Acquisition  Management 

The  Department’s  procedures  for  the  detailed  planning  and  management  of 
acquisition  programs,  when  followed,  and  when  accompanied  by  stable  funding,  have 
generally  produced  good  results.  Any  significant  change  to  the  planned  funding 
profile — almost  always  a  reduction — is  usually  disruptive  to  even  the  best-planned  and 
most  technically  stable  program.  The  resulting  program  stretches  usually  increase  total 
program  cost.  Nevertheless,  such  “routine”  funding  instability  is  not  a  major  contributor 
to  the  excessive  cost  growth  of  the  minority  of  DoD  acquisition  programs  that  contribute 
to  the  large  majority  of  the  Department’s  reported  total  acquisition  cost  growth. 

This  report  identifies  five  funding  stability  issues  for  consideration.  Of  those,  only 
the  following  two  are  deemed  worthy  of  a  near-term  USD(AT&L)  initiative.  The  other 
three  issues  (“fencing”  funds — not  worthwhile;  provisions  for  unplanned  programs  and 
milestone  budgeting — both  excessively  difficult)  are  discussed  in  the  body  of  the  report 

1.  Funding  Stability  Initiative  1:  Mitigation  of  the  Migration  of  Planned 
Acquisition  Funds  to  the  Operating  Accounts 

The  mismatch  between  the  Department’s  ability  to  carefully  forecast  the  funds 
needed  in  the  future  for  each  approved  weapon  system  and  the  Department’s  ability  (or 
willingness)  to  forecast  its  needs  for  future  operating  funds  is  an  important  contributor  to 
the  instability  of  acquisition  program  funding.  Every  year  the  Department  is  faced  with 
the  need  to  cut  back  on  previously  planned  and  approved  acquisition  spending  in  order  to 
meet  unplanned  needs  for  operating  and  maintenance  (O&M)  funds  and  military 
personnel  (MILPERS)  costs.3 

This  source  of  instability  in  acquisition  program  funding  is  chronic — not  just 
associated  with  the  current  high  tempo  of  unplanned  operations.  Indeed,  a  former  Deputy 
Secretary  of  Defense  recently  characterized  DoD’s  long-standing  budgeting  practices  for 
O&M  and  MILPERS  as  knowingly  “dishonest.”4 

A  broad  solution  to  this  problem  would  be  for  the  Department  to  plan  its  long¬ 
term  O&M  spending  to  the  same  “most  likely  cost”  criteria  that  it  tries  to  apply  to  the 


3  See  the  testimony  of  former  Deputy  Secretary  of  Defense  John  Hamre  to  the  Airland  Subcommittee  of 
the  Senate  Armed  Services  Committee  on  15  November  2005. 

4  Ibid. 


ES-3 


planned  cost  of  its  weapons  systems.  The  Team  believes  that  real  progress  will  occur  in 
this  area  only  if  the  Secretary  himself  demands  it. 

Recommendation : 

Although  the  DAE  has  little  formal  leverage  over  the  budgeting  of  operating, 
maintenance,  and  military  personnel  funds,  the  USD(AT&L)  should  enlist  the  Secretary’s 
support  in  effecting  the  needed  improvement.  The  misperception  that  most  acquisition 
cost  growth  results  from  poor  acquisition  management  reflects  badly  on  overall 
Departmental  oversight. 

2.  Funding  Stability  Initiative  2:  The  Need  for  Planned  “Reserves” 

The  lack  of  DoD  “Management  Reserves”  is  a  known  source  of  program 
instability.  Managers  of  complex  civil  projects  know  that  they  cannot  accurately  predict 
the  total  cost  just  by  adding  up  the  costs  of  each  of  the  initially  planned  steps  known  to  be 
needed.  In  an  analogous  situation,  builders  promise  a  result  at  a  fixed  price  but  always 
include  some  unallocated  contingency  reserves  in  their  bids  based  on  their  professional 
experience,  or  they  would  soon  be  bankrupt. 

In  sharp  contrast  to  private  sector  practice,  the  major  segment  of  the  DoD 
acquisition  spending  that  is  devoted  to  Research,  Development,  Test  and  Evaluation 
(RDT&E),  and  most  of  the  remainder  that  goes  to  procurement  spending,  is  contracted 
not  on  a  fixed-price  basis,  but  using  cost-type  contracts.  In  principle,  the  Department 
could — and  should — include  prudent  contingency  reserves  in  the  estimates  of  both 
development  and  procurement  costs.  In  practice,  however,  there  is  a  bias  against  such 
prudent  planning,  in  large  part  because  there  is  always  some  need  that  is  more  tangible 
than  the  unknowns  that  motivate  planning  for  reserves. 

In  past  years  the  Department  has  tried  several  different  approaches  to  establishing 
prudent  levels  of  reserves.  In  all  cases,  such  schemes  have  been  abandoned  after  only  a 
few  years,  or  even  months,  because  of  the  difficulty  in  holding  back  funds — even  outyear 
funds — against  unknown  eventualities.  The  Team’s  sense  is  that  the  stability  of 
acquisition  programs  is  important  enough  to  warrant  yet  another  try  at  establishing  such  a 
reserves  program — not  necessarily  for  the  budget  year,  but  surely  for  the  outyear  plan. 
Independent  cost  estimates,  important  as  they  are,  are  not  intended  to  substitute  for 
careful  “bottom-up”  cost  estimates  by  government  program  management  officials. 
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Recommendation : 


The  USD(AT&L)  should  takes  steps  to  require  that  a  prudent  level  of  acquisition 
program  management  reserves  be  included  in  the  outyears  of  the  08-13  FYDP,  when 
completed  in  early  2007.  The  extent  to  which  such  reserves  should  be  held  at  the 
Program  Executive  Officer  (PEO),  Service  Acquisition  Executive  (SAE,)  or  Defense 
Acquisition  Executive  (DAE)  level  should  be  assessed  as  a  matter  of  priority. 

C.  Joint  Acquisition  Programming 

While  this  report  echoes  many  of  the  themes  of  the  Packard  commission,  the  need 
for  much  more  “jointness,”  or  collaborative  acquisition  among  the  Services,  in  the 
establishment  and  execution  of  DoD  acquisition  programs  is  largely  a  post-Packard 
realization.  Under  many  conditions,  better  joint  acquisition  can  result  in  improved 
battlefield  effectiveness  and  reduced  fratricide  through  better  interoperability  as  well  as 
cost  reduction  through  economies  of  scale.  The  major  impediments  to  jointness  in  the 
Department  of  Defense  acquisition  program  planning  are  well  known:  (1)  the  lack  of 
joint  operating  concepts  and  (2)  the  lack  of  joint  funding  mechanisms.  These 
impediments  to  achieving  the  long-standing  goal  of  having  acquisition  programs  “born 
joint”  are  discussed  below. 

1.  Joint  Operating  Concepts 

A  common,  unified  concept  of  how  the  services  intend  to  operate  together  is 
essential  to  real  joint  acquisition.  The  Marines  pride  themselves  on  having  established 
effective  operational  concepts  for  air-ground  operations  across  a  broad  range  of  combat 
scenarios.  The  unity  of  such  concepts  is  widely  advertised  as  integral  to  their  Marine  Air- 
Ground  Task  Force  (“MAGTF”)  organizational  structures.  Joint  operating  concepts  and 
joint  integrating  concepts  that  cross  the  boundaries  of  the  other  Services,  however,  are 
much  less  well  developed.  The  Joint  Staff  is  working  to  develop  a  broad  range  of  joint 
operating  and  integrating  concepts  to  address  this  need,  but  progress  is  glacial. 

Research  at  IDA  into  the  differences  between  the  originally  joint  F-lll  program 
that  reverted  to  a  single-Service  program,  and  the  largely  successful  (to  date)  joint 
F-35  program,  identified  one  overriding  factor.  In  the  F-lll  program  the  Navy  and  Air 
Force  failed  to  agree  on  an  acceptable  set  of  joint  perfonnance  requirements,  largely 
because  of  differing  operating  concepts.  In  contrast,  agreement  was  reached  early 
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between  the  Services  on  F-35  performance  requirements,  thereby  permitting  considerable 
commonality. 

The  lack  of  planned  interoperability  among  the  military  services  is  not  a  new 
problem.  In  discussing  the  acquisition  of  equipment  important  to  the  interoperability  of 
US  forces,  the  2004  CSIS  report  Beyond  Goldwater-Nichols  (BG-N)  stated: 

This  enduring  lack  of  jointness  in  how  DoD  procures  weapons  has  both  raised 
the  cost  of  military  operations  (e.g.,  persistent  interoperability  problems  cause 
friendly  fire  casualties)  and  constrained  the  growth  of  US  military  capabilities 
(e.g.,  Services  invest  too  much  in  duplicative  capabilities  and  too  little  in  Low 
Density/High  Demand  assets). 

Nowhere  is  the  need  for  improved  coherence  in  the  acquisition  of  military 
capabilities  more  apparent  than  in  command,  control,  and  communications  (C3)  systems. 
Although  the  Under  Secretary  for  Acquisition,  Technology  and  Logistics  has  both  the 
responsibility  and  authority  to  decide  what  the  Department  will  propose  to  buy,  he  is  not 
chartered  or  staffed  to  establish  the  joint  operating  concepts  that  are  particularly 
important  to  the  acquisition  of  C3  systems. 

Joint  acquisition  opportunities  too  often  become  apparent  only  very  late  in  the 
process  of  establishing  new  service-unique  acquisition  programs.  A  recent  example 
involved  the  DAE  exercising  his  authority  by  delaying  an  Anny  plan  to  issue  an  RFP  for 
a  new  tactical  airlifter  t  in  order  to  provide  time  to  establish  a  joint  Army/Air  Force 
program  for  acquiring  a  common  aircraft. 

Secretary  Krieg  has  reportedly  moved  to  take  a  greater  role  in  the  examination  of 
requirements.  This  is  true  not  only  for  major  defense  acquisition  programs  (MDAPs),  but 
also  for  programs  that  may  not  meet  the  MDAP  cost  thresholds  but  are  vital  enablers  to 
future  joint  military  operations,  such  as  command  and  control  systems.  This  latter 
consideration  reinforces  the  Team’s  belief  that  a  greater  degree  of  unity  could  be 
achieved  in  DoD’s  acquisition  management  practices  if  the  command  and  control-related 
acquisition  programs  now  structured  and  overseen  by  the  Assistant  Secretary  of  Defense 
(Network  and  Information  Integration)  [ASD  (Nil)],  the  Commander,  Joint  Forces 
Command,  and  the  Joint  Staff  (J-6),  were  more  clearly  brought  within  the  purview  and 
processes  of  the  USD(AT&L). 

Similarly,  the  Team  notes  the  lack  of  a  management  mechanism  for  examining 
the  prospects  for  “joint”  Special  Acquisition  Programs  (SAPs).  At  present  it  appears  that 
the  Services  each  establish  and  manage  their  own  SAPs,  and  control  access  thereto,  in 
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ways  that  largely  preclude  consideration  of  joint  applications.  As  with  acquisition 
programs  now  overseen  by  Nil,  the  Department’s  Special  Access  Programs  would  likely 
benefit  from  greater  oversight  by  the  USD(AT&L),  with  early  attention  given  to  potential 
joint  applications. 

Recommendation : 

In  support  of  the  previous  recommendation  that  the  USD(AT&L)  exert  stronger 
leadership  in  structuring  and  conducting  the  broad  AoA  that  should  precede  any 
Milestone  A  review  of  a  potential  MDAP  or  important  joint  enabling  program,  the 
USD(AT&L)  should  change  existing  procedures  such  that  all  C3  and  SAP  programs  that 
are  potentially  important  to  future  joint  capabilities  are  subjected  to  his  early  review.  To 
this  end  he  should  also  consider  reorganizing  the  OUSD(AT&L)  staff  such  that  a  new 
element  is  dedicated  to  detecting  and  analyzing  potential  cross-Service,  or  cross-mission 
joint  applications  of  nascent  Service  development  programs  regardless  of  size  or  stage  of 
formalization.  This  could  build  on  the  recently  instituted  OUSD(AT&L)  Capability  Area 
Review  process. 

2.  Joint  Funding 

A  single  flow  path  for  funding  for  a  joint  program  clearly  helps.  “Purple”  or 
“joint”  funding  of  common  support  activities  has  largely  proven  its  worth.  Such  DoD- 
wide  activities  as  the  Defense  Logistics  Agency,  Defense  Infonnation  Services  Agency, 
Defense  Finance  and  Accounting  Service,  and  the  Defense  Contract  Management 
Agency  are  widely  agreed  to  now  be  working  well  and  are  considerably  less  costly  than 
they  would  have  been  had  the  Services  each  retained  such  functions.  More  such  “joint” 
funding  appears  to  be  a  necessary  condition  for  achieving  real  progress  towards  joint 
acquisition. 

There  are  no  examples  of  single  sources  of  joint  funding  for  successful  major 
cross-Service  weapons  acquisition  programs  that  have  been  fielded.  The  Missile  Defense 
Agency  has  been  a  single  source  of  funding  only  for  the  development  of  ballistic  missile 
defenses,  not  their  production,  fielding,  and  support. 

It  may  take  another  Herculean  effort,  such  as  that  that  went  into  the  Goldwater- 
Nichols  Act  itself,  to  move  the  Department  into  a  bold  new  approach  to  acquiring 
capabilities  that  are  truly  “born  joint.”  It  seems  unrealistic  to  expect  much  more  progress 
toward  improved  joint  acquisition  without  a  major  effort  by  both  the  Secretary  of 
Defense  and  the  Congress.  In  the  meantime,  the  USD(AT&L)  could  take  the  lead  in 
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establishing  a  joint  funding  and  management  chain  for  C3  programs  important  to  future 
joint  operations. 

Recommendation : 

The  USD(AT&L)  should  take  the  lead  in  establishing  a  new  joint  C3  acquisition 
management  office  that  pulls  together  the  current  disparate  Departmental  initiatives  in 
this  area.  The  resulting  structure  would  be  centrally  funded  and  managed  in  support  of 
the  joint  operational  commanders. 

II.  IMPROVED  OUTCOMES  THROUGH  BETTER  SYSTEMS 
MANAGEMENT  PRACTICES 

A  major  thread  of  the  Packard  Commission’s  recommendations  was  to  “fly  before 
buy.”  Although  the  importance  of  this  systems  management  concept  is  embedded  in  DoD 
policies,  these  policies  have  been  insufficiently  heeded  in  recent  years. 

Nonetheless,  the  Department  has  relied  heavily  on  three  key  tools  that  are 
intended  to  implement  this  principle:  One  is  working  well  (operational  testing);  the  other 
two — systems  engineering  and  technology  readiness  assessments  (TRA) — are  not.  Each 
is  discussed  below: 

A.  Formal  Operational  Testing 

Significant  benefits  have  accrued  to  our  warfighters  by  independently  ensuring 
that  their  equipment  has  demonstrated  both  operational  effectiveness  and  suitability  in 
the  field.  US  military  equipment  is  the  envy  of  the  world’s  fighting  forces,  in  large  part 
as  a  result  of  this  rigorous  and  independent  testing. 

One  complaint  about  the  current  operational  testing  process  is  that  the  testing 
community  is  too  often  seen  as  testing  systems  against  obsolete  “requirements.”  In  this 
regard,  as  the  USD  increases  his  involvement  in  overseeing  the  requirements,  one 
productive  avenue  of  activity  would  be  to  strengthen  the  process  that  is  intended  to 
ensure  that  the  formal  requirements  documents  and  associated  “test  plans”  are  kept 
current.  Although  this  measure  is  important,  the  Team  does  not  believe  it  warrants  a 
major  DAE  initiative. 
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B.  Adequate  Systems  Engineering  in  Programs 

Good  systems  engineering  is  critical  to  the  successful  development  and 
production  of  complex  weapons  systems.  Cost-performance  trades  are  now  being  made 
too  late  and  immature  technology  is  being  relied  upon  too  often.  As  a  result,  rework 
ensues,  schedules  slip,  and  cost  grows. 

The  general  problem  is  illustrated  in  a  recent  Government  Accountability  Office 
(GAO)  report  that  found  that  at  their  point  of  critical  design  review  (CDR)  less  than  half 
of  DoD’s  major  development  programs  demonstrated  adequate  design  stability,  and  that 
the  cost  of  the  programs  evidencing  such  unstable  designs  grew  about  eight  times  more 
than  did  the  cost  of  the  programs  with  stable  designs.5 

The  consequences  of  such  weak  systems  engineering  involve  not  only  short-term 
cost  growth  and  schedule  slips,  but  also  increased  operating  and  support  costs  after 
weapons  systems  are  fielded.  In  recent  years  there  has  been  a  substantial  drop  in  the 
percentage  of  systems  passing  formal  operational  suitability  testing,  while  at  the  same 
time  nominal  combat  effectiveness  passing  rates  have  increased.  These  trends  are  shown 
in  Figure  ES-1.6 
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Figure  ES-1.  Percentage  of  Systems  Passing  Operational  Test 


Declining  suitability  numbers  and  their  divergence  from  the  effectiveness  scores 
clearly  demonstrate  a  shift  in  emphasis  within  both  the  government  and  contractor 


5  Defense  Acquisitions:  Assessments  of  Selected  Major  Weapon  Programs,  GAO-05-301,  March  2005. 

6  There  has  been  a  trend  of  more  systems  passing  operational  test  from  an  effectiveness  perspective. 
This  trend  is  primarily  the  result  of  a  change  in  testing  philosophy.  Early  on,  tests  were  mostly 
conducted  on  a  pass-fail  basis  against  a  specific  (and  sometimes  outdated  or  arbitrary)  number.  In 
today’s  environment,  testing  is  more  often  based  on  the  ability  to  accomplish  the  mission. 
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program  offices  away  from  achieving  suitability  goals  that  are  highly  dependent  on  sound 
systems  engineering  practices.  The  drop  in  suitability  shown  in  Figure  ES-1  can  also  be 
attributed  to  a  decline  in  systems  engineering  expertise  in  program  offices  based  on  an 
overreaction  to  earlier  “acquisition  reform”  initiatives.  A  number  of  parallel  industry  and 
DoD  studies  have  been  conducted  to  identify  the  specific  systems  engineering  concerns 
that  need  to  be  addressed.  They  all  identified  broad  systems  engineering  weaknesses  in 
the  government’s  program  offices  as  a  major  problem. 

In  confirmation  of  the  foregoing  observations,  a  recent  NDIA  survey  asked 
industry  members  the  question,  “IIow  would  you  rate  the  quality  and/or  effectiveness  of 
the  [government]  customer’s  program  manager  systems  engineering  capability ?”  The 
synthesized  answer  was:  “ Presently ,  even  within  a  given  Sendee,  this  varies  significantly 
from  program  to  program.  There  is  a  decided  lack  of  systems  engineering  capability  and 
understanding  compared  with  pre-acquisition  reform  days.  ”  (emphasis  added) 

The  Team  believes  that  the  observed  deterioration  in  DoD  systems  engineering 
discipline  is  an  unintended  consequence  of  two  key  tenets  of  the  Acquisition  Reform 
efforts  of  the  1990s:  (1)  increased  reliance  on  the  defense  industry  itself  to  oversee 
complex  acquisition  programs,  and  (2)  reduced  government  oversight  activities  that 
industry  believed  led  to  increased  cost  and  schedule  slippages. 

As  part  of  the  attempt  to  reduce  oversight  practices  that  industry  found  intrusive, 
many  new  development  contracts  eschewed  the  traditional  requirements  for  some  types 
of  fonnal  reliability  and  supportability  pre-production  testing.  The  resulting  lack  of 
contractual  requirements  for  testable  “suitability”  measures  appears  to  have  contributed 
directly  to  the  deterioration  of  “suitability”  in  operational  testing  as  shown  in  Figure  ES-1 

If  all  of  the  systems  engineering  management  conditions  for  a  successful  program 
were  to  be  met  before  formal  program  initiation  at  Milestone  B,  program  managers 
would  be  in  a  much  better  position  to  maintain  cost  and  schedule.  This  contrasts  sharply 
with  the  current  situation  in  which  systems  engineering  plans  are  hastily  drafted  just  prior 
to  Milestone  B,  are  frequently  found  to  be  inadequate,  and  do  not  inform  or  in  some  cases 
even  reflect  the  important  multi-year  risk  reduction  efforts  that  should  have  been  started 
at  Milestone  A  in  order  to  achieve  acceptable  levels  of  technical  risk  by  Milestone  B. 
Under  current  practice,  real  oversight  of  MDAPs  by  the  USD(AT&L)  doesn’t  currently 
begin  until  Milestone  B  is  imminent. 
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Recommendations : 


The  Team  recommends  three  actions  that  will  strengthen  the  Department’s 
systems  engineering  management  practices. 

First,  in  conjunction  with  the  revitalized  Analysis  of  Alternatives  process 
recommended  in  Section  1,  the  USD(AT&L)  should  reinstitute  Milestone  Decision 
Authority  oversight  of  the  Milestone  A  process  so  that  a  sound  systems  engineering 
management  plan  is  put  in  place  for  each  alternative  that  will  be  pursued  into  a  fonnal 
risk  reduction/demonstration/validation  phase  leading  to  a  possible  Milestone  B  formal 
initiation  of  a  major  new  acquisition  program  (or  joint  enabling  C3  or  SAP  program). 

Second,  there  should  be  a  very  high  bar  for  waivers  of  an  inadequate  systems 
engineering  plan  at  formal  program  initiation  at  Milestone  B.  OSD  scrutiny  of  programs 
should  be  much  more  independent  and  thorough  for  the  events  leading  up  to  program 
initiation.  Specifics  from  each  program’s  systems  engineering  plan  should  be 
incorporated  into  the  Annual  Operating  Plan. 

Third,  industry  should  be  better  incentivized  to  meet  suitability  goals.  Most 
importantly,  appropriate  suitability  demonstrations  should  be  made  co-equal  with  other 
key  systems  performance  requirements  in  the  prime  contracts. 

The  net  result  of  these  three  recommendations  would  be  an  overall  reduction  in 
OSD  oversight  and  a  shift  of  staff  effort  from  trying  to  help  salvage  troubled  programs  to 
better  planning  and  executing  programs  in  their  early  stages  (including  joint  programs). 

C.  Technical  Readiness  Assessments 

Unambiguous  DoD  policy  and  guidance  requires  that  certain  levels  of  technology 
maturity  be  demonstrated  before  formal  acquisition  program  initiation  at  Milestone  B.  It 
is  widely  acknowledged  that  this  guidance  is  too  often  being  waived.  Programs  continue 
to  suffer  cost  and  schedule  growth  as  a  result  of  proceeding  into  system  development 
before  key  technologies  are  sufficiently  mature.  The  Army’s  FCS  program  is  a  well- 
known  specific  example. 

Acquisition  executives  frequently  waive  these  clear  requirements  because,  for  one 
thing,  under  current  DoD  oversight  practices,  the  specific  technical  risks  faced  by 
emerging  programs  have  very  little  visibility  prior  to  being  considered  at  the  point  of 
formal  program  initiation — Milestone  B.  At  that  point  program  schedules  and  outyear 
funding  have  been  established  in  such  a  way  that  Milestone  B  disapproval  because  of 
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technical  immaturity  would  seriously  disrupt  the  Service’s  plans,  including  the  award  of 
the  SDD  contract.  A  more  formal  requirement  for  early  identification  and  review  of 
technical  risks  starting  at  Milestone  A  would  significantly  ameliorate  this  problem. 

The  GAO  has  estimated  the  effect  of  such  departures  from  policy  in  an  analysis  of 
54  major  programs.7  According  to  the  GAO,  only  15%  of  these  programs  began  full 
systems  development  with  mature  technology.  Those  that  did  suffered  less  than  one- 
fourth  the  cost  growth  of  the  immature  programs. 

In  addition  to  the  late  involvement  of  the  Milestone  Decision  Authority,  the 
ability  of  DoD  to  competently  identify  critical  technologies,  program  funds  to  reduce 
their  risk,  and  then  assess  their  maturity  at  Milestone  B  is  hobbled  by  the  lack  of 
independent  technical  expertise  within  the  government. 

Recommendation : 

In  support  of  the  increased  USD(AT&L)  responsibility  for  early  program 
definition,  require  that  the  formal,  independent,  identification  of  key  technical  risks  be 
made  at  or  shortly  after  Milestone  A,  with  specific  funded  plans  to  be  put  in  place  to 
achieve  their  specified  level  of  technical  maturity  by  Milestone  B,  as  determined  by 
independent  assessors. 

III.  BETTER  ACOUNTABILITY  IN  THE  ACQUISITION  WORKFORCE 

In  its  simplest  tenns,  the  Packard  Commission  recommended  that  DoD  implement 
an  industry  model  with  six  attributes.  A  general  assessment  of  the  lack  of  full 
implementation  of  those  recommendations  was  included  as  Appendix  B  in  our  Phase  I 
Report,  IDA  Document  D-3189.  The  following  two  attributes  appear  to  the  Team  to 
warrant  near-term  Under  Secretary  attention  to  help  improve  accountability  in  the 
Department’s  acquisition  process.8 

A.  Clear  Command  Channels 

The  Packard  commission  sought  to  achieve  clear  command  channels  by  limiting 
intervening  layers  between  the  program  manager  (PM)  and  the  Acquisition  decision 


7  Defense  Acquisitions:  Assessments  of  Selected  Major  Weapon  Programs,  GAO-05-301,  March  2005. 

8  Accountability  is  the  “Liability  to  give  account  of,  and  answer  for,  discharge  of  duties  or  conduct; 
responsibility.”  (Oxford  Dictionary) 
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maker  (Defense  or  Service  Acquisition  Executive — DAE  or  SAE)  to  no  more  than  two. 
In  Packard’s  view,  the  shortened  decision-making  chain  would  alter  the  nature  of 
oversight  in  a  way  that  would  reduce  the  number  of  staff  overseers  and  speed  up  decision 
making. 

While  DoD  policy  documents  say  the  Department  has  implemented  Packard,  one 
has  simply  to  overlay  a  graphic  of  the  proposed  Packard  chain  of  command  with  the 
Integrated  Product  Team  (IPT)  structure  currently  mandated  by  DoDI  5000.2  to  see  the 
problem.  Clearly  short  lines  of  communication  have  not  been  established,  and  this  failure 
is  a  matter  of  official  policy.  The  Overarching  IPT  (OIPT)  is  widely  seen  as  a  particular 
impediment  to  the  rapid  and  effective  decision  making  envisioned  by  Packard.  By 
increasing  the  accountability  of  the  subordinate  acquisition  executives,  the  need  for  this 
extra  layer  of  virtual  management  could  be  reduced  or  eliminated.  The  Team’s 
recommendation  for  dealing  with  this  aspect  of  the  accountability  problem  is  reflected  in 
the  subsequent  paragraphs.  This  assessment  is  not  meant  to  include  the  cost  estimating 
and  testing  IPTs,  which  are  working  relatively  well  and  are  grounded  in  law  as  well  as  in 
DoD  Instructions. 

B.  Clear  Responsibility  and  Consequences  for  Actions 

Packard  stated,  “[Ajauthority  for  executing  acquisition  programs — and 
accountability  for  their  results — has  become  vastly  diluted.” 

We  conclude  that  not  only  has  this  problem  not  been  fixed  in  the  19  years  since 
Packard,  it  may  have  gotten  worse  in  some  ways  despite  implementation  of  Packard’s 
recommendation  to  establish  a  Defense  Acquisition  Executive. 

Paul  Beach,  in  his  1990  report  on  how  the  Navy’s  A- 12  stealth  aircraft 
development  program  reached  the  point  where  it  was  cancelled  for  cause,9  itself  a  rare 
instance  of  accountability,  described  the  issue  as  follows: 

The  fundamental  problem  ...  is  to  create  appropriate  incentives  to  enable  senior 
leaders  to  rely  upon  responsible,  accountable  line  managers  for  realistic 
perspectives  on  the  cost,  schedule  and  technical  status  of  their  programs.  Only 
by  doing  so  can  we  increase  efficiency  and  accountability  while  reducing  the 
burdens  imposed  by  undue  regulation  and  stifling  supervision. 


9 


The  continuing  litigation  costs  of  this  disputed  cancellation  have  now  reportedly  exceeded  the  amount 
in  dispute. 
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Unless  means  can  be  found  to  solve  this  abiding  cultural  problem,  the  failures 

evidenced  in  this  report  can  be  anticipated  to  occur  again  in  the  same  or  similar 

form.10 

When  the  acquisition  workforce,  including  middle  and  upper  management, 
perceives  there  are  limited,  if  any,  consequences  for  following  or  not  following  certain 
policies,  they  subordinate  those  policies  to  other  more  immediately  consequential 
considerations.  Such  considerations  can  include  expected  rewards  for  avoiding  taking 
management  actions  that  would  imply  schedule  slips,  or  that  would  trade  minor 
performance  goals  favored  by  senior  service  executives  against  cost. 

A  major  reason  for  this  current  disconnect  between  the  fonnal  acquisition 
reporting  chain  and  true  accountability  that  acquisition  executives  are  not  the  primary 
conduit  for  long-  or  short-term  program  funding.  This  is  in  direct  contrast  to  industry 
practice  where  a  program  manager’s  agreement  with  his  leadership  usually  includes 
funding  guarantees. 

C.  Initiatives  for  Improved  Accountability 

1.  Establish  an  Annual  Operating  Plan  for  each  acquisition  program 

The  basic  thrust  of  this  section  is  that  the  Under  Secretary  of  Defense  (AT&L) 
should  establish  a  new  oversight  process  that  utilizes  tiered  accountability.  A  more 
accountable  acquisition  management  system  could  best  be  centered  on  an  annual 
operating  plan.  This  would  largely  avoid  the  problem  noted  above  in  which  non¬ 
acquisition  officials  have  authority  over  longer-term  funding. 

An  Annual  Operating  Plan  (AOP)  based  on  approved  1-year  funding  could 
establish  and  measure  progress  on  those  events  in  the  next  12  months  that  are  on  or  near 
the  critical  path.  If  the  AOP  is  based  on  the  fiscal  year,  most  funding  changes  and 
impacts  are  known  by  the  August/September  time  frame,  which  provides  sufficient  time 
to  set  the  next  year’s  goals  in  a  manner  that  is  almost  entirely  under  the  control  of  the 
CAE,  PEO,  and  PM. 

The  establishment  of  an  Annual  Operating  Plan  would  more  properly  align 
responsibilities  and  authority  within  the  Department’s  acquisition  process,  and  pennit 


10  Chester  Paul  Beach,  Jr.,  “A-12  Administrative  Inquiry,”  28  November  1990. 
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CAEs,  PEOs  and  PMs  to  become  more  objectively  accountable  for  the  progress  of  their 
programs. 


2.  Establish  A  More  Flexible  and  Accountable  Acquisition  Workforce 

DoD  needs  an  acquisition  workforce  that  is  well  trained,  broadly  experienced,  and 
mobile.  The  Defense  Acquisition  Workforce  Improvement  Act  (DAWIA)  establishes  a 
reasonable  starting  point,  but  more  improvements  are  needed.  In  particular,  management 
must  be  able  to  unilaterally  deploy  personnel  where  their  talents  and  backgrounds  are 
most  needed.  In  addition,  management  needs  to  be  actively  engaged  in  career  planning 
to  ensure  a  deep  pool  of  the  best  talent  available.  Management  should  adopt  for  the 
civilian  acquisition  workforce  some  of  the  characteristics  of  the  Service’s  officer 
management  programs.  That  does  not  have  to  mean  an  up-or-out  policy,  but  it  may  mean 
an  up-or-stav  put  policy. 

On  the  other  hand,  if  the  current  practice  of  rotating  military  program  managers 
too  frequently  cannot  be  ameliorated,  serious  consideration  should  be  given  to  staffing 
most  DoD  program  offices  with  civilian  program  managers  who  will  be  expected  to  stay 
in  place  until  the  program  phase  succeeds  (or  is  cancelled).  Military  officers  could  bring 
their  valuable  battlefield  experience  to  bear  as  rotating  deputy  program  managers. 

Recommendations: 

•  The  Component  Acquisition  Executives  (CAE)  should  be  held  responsible  and 
accountable  for  their  programs. 

•  DAE  staff  should  be  responsible  for  and  given  sufficient  lead  time  and 
information  by  the  CAE  to  prepare  the  DAE  for  the  review,  but  not  for  deciding 
when  and  whether  to  hold  a  review. 

•  The  Under  Secretary  of  Defense  (AT&L)  should  use  the  flexibility  inherent  in  the 
new  National  Security  Personnel  System  to  exercise  aggressive  leadership  in 
career  management,  creating  a  truly  professional  acquisition  corps  that  is  well  and 
broadly  trained,  and  easily  deployed  on  an  individual  basis  to  where  the  skills  are 
needed. 

•  Qualification  requirements  for  PM  selection  should  be  more  strictly  enforced,  as 
should  the  goals  for  keeping  PMs  in  place  through  the  full  current  phase  of  their 
program. 

The  foregoing  changes  would  permit  the  Under  Secretary  of  Defense  (AT&L)  to  cut  back 
on  the  size  and  function  of  the  OIPT  structure,  thereby  freeing  up  resources  that  could  be 
applied  to  improving  the  technical  readiness  assessments  and  joint  acquisition  processes 
as  recommended  earlier. 
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IV.  BETTER  ACCESS  TO  TECHNOLOGY 


The  one  area  in  this  sector  in  which  the  Team  believes  a  new  initiative  might  be 
warranted  is  the  needed  amelioration  of  the  decline  of  the  US  as  the  major  source  of  new 
technologies  of  potential  future  importance  to  the  Department  of  Defense.  This  decline 
stems  from  the  globalization  of  science  and  technology  research,  the  relative  stagnation 
of  DoD  funding  of  U.S.  science  and  technology,  the  overall  decline  of  federal  funding  in 
this  area,  and  the  sharp  decline  in  investment  in  long-term  science  and  technology  by 
most  major  defense  contractors. 

The  Team  believes  that  a  minor  initiative  is  warranted  that  would  reexamine  the 
decision  that  DoD  made  in  the  1990s  to  decouple  the  allowability  of  industrial 
independent  research  and  development  (IR&D)  spending  in  the  overhead  charges  billed 
to  government  contracts  from  any  formal  evaluation  of  the  potential  value  of  such 
spending.  Resumption  of  DoD  “scoring”  of  the  long-term  potential  of  such  spending  as  a 
factor  in  determining  the  extent  to  which  it  would  be  reimbursed  by  the  government 
would  likely  reverse  the  decline  in  long-term  industrial  science  and  technology 
investments.  It  would  also  help  government  technical  experts  become  and  remain  more 
fluent  in  emerging  technical  developments. 

Recommendation : 

DoD  should  conduct  a  short  in-house  evaluation  of  this  restoration  concept,  and, 
if  this  Team’s  assessment  is  sustained,  promulgate  the  appropriate  changes  to  DoD 
directives  and  accounting  procedures. 
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BACKGROUND  AND  OVERVIEW 


The  Institute  for  Defense  Analyses  (IDA),  in  cooperation  with  the  CNA 
Corporation  (CNAC) — the  Team — was  tasked,  first,  to  review  and  assess  the  current 
status  of  the  many  acquisition  initiatives  that  have  been  recommended  and/or  attempted 
in  recent  years  and,  second,  to  identify  and  analyze  a  subset  of  initiatives  that  the  team 
finds  to  have  potential  for  near-term  management  emphasis  that  could  provide  visible 
improvements  to  the  much-criticized  Defense  acquisition  system. 

The  Team  is  composed  of  several  experienced  senior  managers  and  analysts  who 
have  had  extensive  experience  in  the  management  and  oversight  of  defense  acquisition 
programs  at  the  OSD,  Service,  and  industry  levels.  Membership  of  the  Team  is 
documented  at  Appendix  A. 

The  results  of  Phase  I  were  briefed  to  the  sponsors  on  25  September  2005,  and  the 
written  Phase  One  report  was  subsequently  provided  to  the  sponsors  as  IDA  Document 
D-3189.  The  present  report  documents  the  results  of  the  second  and  final  phase  of  the 
project. 

The  Team  systematically  reviewed  the  results  of  the  Phase  I  research  in  order  to 
identify  those  specific  initiatives  worthy  of  early  consideration  by  the  Defense 
Acquisition  Executive.  Coincidentally,  two  members  of  the  Team  (Gene  Porter  and  Gary 
Christie)  were  invited  to  testify  to  the  AirLand  Subcommittee  of  the  Senate  Anned 
Services  Committee  on  15  November  2005  on  problems  in  defense  acquisition 
management.  Preparation  of  that  testimony  provided  an  additional  near-term  focus  for  the 
Team’s  work  on  this  Phase  II  report,  which  is  generally  consistent  therewith. 

A.  TWO  MAJOR  ACQUISITION  MANAGEMENT  ISSUES  EXCLUDED 

The  Team  did  not  review  or  assess  the  several  ongoing  efforts  to  provide 
increased  authority  and  more  responsive  processes  for  rapid  acquisition  of  key  items  of 
material  needed  to  support  current  military  operations.  Our  review  focused  on  the 
processes  and  procedures  for  the  long-term  acquisition  of  military  equipment  and  systems 
essential  for  properly  equipping  and  supporting  the  future  military  force  structure. 
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Additionally,  the  Team  did  not  explicitly  address  the  issue  of  perceived  excess 
cost  growth  in  major  weapons  systems  other  than  by  implication  in  the  areas  of  funding 
stability,  requirements  creep,  and  improved  systems  management.  A  separately  tasked 
study  at  IDA  is  explicitly  investigating  the  cost  growth  issue. 

B.  INITIATIVES  REVIEWED  AND  RECOMMENDED  FOR  FURTHER 
USD(AT&L)  ACTION 

This  Phase  II  report  first  focuses  on  three  major  initiatives  and  then  concludes 
with  a  review  of  a  fourth  minor  initiative: 

1.  Better  integration  of  Requirements/Resources/Acquisition  management 
processes  into  a  system  that  provides  for  both  better  Joint  outcomes  and 
greater  funding  stability  (Section  I). 

2.  Potential  improvements  to  in-house  systems  management  (section  II). 

3.  Strengthen  accountability  in  the  acquisition  management  chain 
(section  III). 

4.  Improved  access  to  technology  (section  IV). 
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I.  IMPROVED  PROGRAM  STABILITY  AND  “JOINTNESS” 

OUTCOMES  THROUGH  BETTER  INTEGRATION  OF 
REQUIREMENTS/PROGRAMMING/ACQUISITION  PROCESSES 


One  dominant  theme  of  our  research  is  that  the  Packard  Commission  had  the 
important  “what  to  buy”  management  principle  about  right,  but  that  those  elements  of  the 
Packard  Commission  recommendations  that  would  have  empowered  the  Under  Secretary 
of  Defense  [USD(AT&L)]  to  effect  such  a  change  were  never  fully  implemented.1 

But  one  major  emergent  challenge  that  has  proved  to  be  important  to  defense 
acquisition  management  practice  was  missing  from  the  earlier  Packard  recommendations: 
the  widely  recognized  need  to  improve  policies  and  procedures  that  encourage  more  and 
better  “joint”  acquisition.  Lack  of  progress  toward  joint  acquisition  is  exemplified  by  the 
lack  of  a  formal  DoD  joint  acquisition  management  structure,  except  at  the  Special 
Operations  Command  (SOCOM)  and  at  the  Defense  Agencies. 

The  difficulty  in  getting  good  “joint”  outcomes  from  the  DoD  acquisition  process 
is  directly  related  to  the  weak  integration  of  the  Department’s  weapons  system 
requirements  process,  the  formal  acquisition  management  process,  and  the 
programming/budgeting  process. 

The  process  integration  problem  and  the  associated  funding  instability  will  be 
addressed  first,  and  joint  acquisition  later  in  this  section.  These  problems  together 
underlie  many  of  the  major  current  issues  on  acquisition  management.  The  problem  of 
the  excessive  cost  growth  of  the  minority  of  DoD  acquisition  programs  that  contribute  to 
the  majority  of  overall  cost  growth  is  being  addressed  in  detail  in  a  separate  ongoing  IDA 
study.  Most  such  cases  of  excessive  procurement  cost  growth  resulted  from  causes  other 
than  instabilities  in  funding. 


1  This  discussion  of  this  issue  is  supplemented  by  a  legal  analysis  of  the  USD(AT&L)  authority  in  such 
matters  that  was  submitted  as  Appendix  B  in  the  Phase  I  Report,  IDA  Document  D-3189. 
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A.  INTEGRATION  OF  DOD  PROCESSES:  THE  “REQUIREMENTS” 
PROBLEM 

1.  Requirements  Formulation 

A  key  step  in  sound  acquisition  program  planning  is  to  establish  realistic, 
achievable,  and  affordable  statements  of  the  intended  principal  characteristics  of  a 
notional  new  weapon  system.  There  is  an  ongoing  tendency  in  some  quarters  to  consider 
the  final  establishment  of  such  “requirements”  as  quite  separate  from  the  acquisition 
process  itself — a  distinction  that  is  unfortunately  reflected  in  current  DoD  Instructions. 
That  is  not  what  the  Packard  Commission  recommended,  nor  is  it  consistent  with  the 
legislation  that  established  the  position  of  the  Under  Secretary  for  Acquisition.  The 
USD(AT&L),  acting  on  behalf  of  the  Secretary,  has  both  the  responsibility  and  authority 
to  decide  the  key  characteristics  of  the  equipment  the  Department  will  ask  the  President 
to  request  Congress  to  fund. 

It  is  highly  appropriate  that  the  nation's  warfighting  experts,  both  the  Combatant 
Commanders,  and  the  Chiefs  of  the  Military  Services,  be  deeply  involved  in  the 
deliberations  that  lead  to  establishing  a  fonnal  acquisition  program.  Indeed,  the  Service 
Chiefs  in  particular  are  routinely  heavily  involved,  not  only  in  identifying  needs  and 
advising  on  the  requirements  at  the  start  of  programs,  but  also  in  many  of  the  ongoing 
cost-perfonnance  trades  that  are  made  as  a  program  proceeds  through  development  and 
into  production.  Well-known  examples  include  the  Army’s  Future  Combat  System,  the 
Navy’s  Littoral  Combat  Ship,  and  the  Air  Force’s  F-22A  programs.  The  characteristics, 
quantities,  and  timing  of  the  programs  were  each  proposed  by  their  respective  Service 
Chiefs,  who  have  continued  to  be  heavily  involved  in  decisions  on  key  cost-perfonnance 
trades  as  the  systems  proceed  through  development.  The  more  central  a  weapon  system  is 
to  a  Service’s  vision  of  its  own  core  competency  and  preferred  warfighting  doctrine,  the 
stronger  the  involvement  of  the  Service  leadership  in  the  conduct  of  the  acquisition. 

Indeed,  in  some  circumstances  senior  Service  official’s  influence  on 
“requirements”  is  so  pervasive  and  detailed  that  the  Program  Manager  has  insufficient 
flexibility  to  trade  cost  and  performance,  as  discussed  below  and  in  Section  II. 

In  addition  to  the  primary  role  of  the  Services  in  proposing  new  weapons  systems, 
the  Chainnan  of  the  Joint  Chiefs  of  Staff  has  established  a  formal  process  by  which  his 
staff  develops  advice  that  is  provided  to  the  Defense  Acquisition  Executive  (DAE)  as  to 
what  the  Department  should  acquire  to  meet  a  recognized  need — particularly  from  the 


1-2 


viewpoint  of  the  Combatant  Commanders  and  the  joint  warfighting  community.2  But  the 
team  is  aware  of  no  major  new  weapon  system  that  proceeded  into  the  acquisition  process 
after  first  being  proposed  by  a  Combatant  Commander  or  other  joint  entity. 

Regardless  of  the  source  of  the  original  proposal  that  a  new  system  be  acquired,  it 
is  the  DAE  who  ultimately  makes  decisions  on  the  weapons  needed  to  help  meet  the 
Department’s  overall  capabilities  goals.  The  DAE  considers  the  advice  provided  by  the 
members  of  the  Defense  Acquisition  Board — ideally  after  examining  the  results  of  an 
objective,  broad  Analysis  of  Alternatives  that  the  DAE  helped  structure,  and  before  a 
contracting  source  is  selected.  For  this  process  to  work  well  there  needs  to  be  close 
cooperation  between  the  warfighting  “customers”  and  the  acquisition  executive,  as  was 
envisioned  by  the  Packard  Commission’s  unimplemented  recommendation  for  the 
establishment  of  a  Joint  Requirement’s  Management  Board  (JRMB). 

Even  before  decisions  are  made  on  specific  system  performance  objectives,  there 
also  needs  to  be  close  cooperation  between  the  Acquisition  Executive  and  the  other 
members  of  the  Secretary’s  planning  and  programming  community — principally 
PA&E — to  ensure  that  an  appropriately  broad  range  of  alternatives  is  fully  considered — a 
process  that  has  not  always  occurred  in  recent  years. 

In  addition  to  the  sometimes  lack  of  “due  process”  in  establishing  the  analytic 
basis  for  a  new  acquisition  program,  once  a  new  program  is  started,  as  noted  above,  the 
detailed  performance  requirements  and  other  characteristics  are  sometimes  still  specified 
in  such  detail  by  senior  Service  officials  that  the  program  manager  (PM)  has  little  room 
for  making  the  sort  of  cost-perfonnance  trades  for  which,  most  agree,  the  PM  needs  both 
authority  and  latitude  to  decide  for  costs  to  be  adequately  controlled.  The  degree  of  such 
cost-performance  flexibility  that  has  been  provided  to  the  F-35  Joint  Strike  Fighter 
program  managers  is  a  rarity. 


2  The  JCIDS  process  originated  as  an  attempt  to  modernize  the  process  for  developing  “required” 
weapon  system  performance  characteristics.  The  JCIDS  schema  is  being  misunderstood  in  that  its  use 
of  the  term  “approve  and  validate”  rather  than  “advise  and  assist  the  CJCS”  is  causing  confusion  about 
and  proliferating  the  idea  that  requirements  decisions  are  uniquely  military  and  are  separated  from 
civilian  acquisition  decisions.  In  addition,  the  JCIDS  process  is  now  cast  as  a  process  for  defining  the 
overall  capabilities  needed  by  the  entire  Defense  Department,  as  seen  by  the  Joint  Staff.  As  such,  it  is 
not  at  all  clear  how  this  broad,  fiscally  unconstrained,  military  look  at  100%  of  the  defense  program 
should  routinely  interface  with  the  acquisition  process  that  deals  with  the  25%  of  the  defense  program 
allocated  to  weapons  systems. 
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Secretary  Kenneth  Krieg  and  two  of  the  Service  Acquisition  Executives  recently 
testified  that  they,  along  with  their  military  counterparts,  the  Vice  Chairman  of  the  Joint 
Chiefs  of  Staff,  and  the  Service  Chiefs,  are  increasingly  involved  in  making  such  cost- 
performance  trades  on  ongoing  major  programs.  As  a  result,  some  performance  goals 
(formerly  “requirements”)  are  reportedly  being  significantly  altered  in  order  to  better 
balance  cost  and  perfonnance  risks.  As  noted  earlier,  involvement  in  such  activities  by 
the  civilian  acquisition  executives  were  an  important  element  of  the  Packard  Commission 
rec  ommendations . 

2,  Requirements  Creep 

The  pejorative  tenn  “requirements  creep”  doesn’t  always  imply  poor 
management.  Indeed,  in  most  major  programs  that  are  expected  to  take  over  a  decade  to 
proceed  through  risk  reduction,  system  design  and  development,  and  initial  production, 
the  government  should  be  open  to  worthwhile  changes  that  may  add  “up  front”  costs,  if 
that  added  cost  can  be  justified  and  funded  without  undue  risk  to  other  capabilities.  Such 
worthwhile  changes  can  include  revised  responses  to  changing  threats,  the  unplanned 
availability  of  new  technology  that  can  lower  future  production  costs,  and  engineering 
changes  that  lead  to  worthwhile  reductions  in  operating  and  maintenance  costs.  Many 
such  changes  are  clearly  in  the  Department’s  interest,  but  they  should  be  made  in  a 
manner  that  demonstrates  their  appropriateness  via  the  appropriate  acquisition 
executive’s  requirements  review  process,  and  they  should  be  clearly  explained  as  prudent 
to  the  Congress. 

In  keeping  with  this  concept,  a  2004  IDA  study  on  the  reported  growth  of  costs  of 
138  defense  programs  attempted  to  differentiate  between  growth  due  to  “decisions”  such 
as  changing  performance  requirements  or  changing  production  rates,  and  growth  due  to 
“mistakes,”  such  as  erroneous  estimates  of  labor  hours  or  material  costs,  or  premature 
entry  into  system  development  before  technical  risks  have  been  reduced  to  the  prescribed 
level  of  acceptability.3  IDA  found  that  about  half  of  the  cost  growth  in  development 
programs  and  one-third  of  the  (smaller)  growth  in  production  programs  was  due  to 
deliberate  “decisions”  rather  than  “mistakes.” 


3  David  McNicol,  Cost  Growth  in  Major  Weapon  Procurement  Programs,  IDA  Paper  P-3832,  2004. 
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B.  THE  INTEGRATION  OF  DOD  PROCESSES:  STABLE  ACQUISITION 
PROGRAM  FUNDING 

The  Department  has  developed  an  elaborate  and  effective  set  of  procedures  for  the 
detailed  planning  and  management  of  acquisition  programs — procedures  that,  when 
followed,  and  when  accompanied  by  stable  funding,  have  generally  produced  good 
results,  given  the  complexity  of  large  DoD  acquisition  programs.  But  the  success  of  the 
“baseline”  plan  is  highly  dependent  on  the  availability  of  the  planned  funding.  Therefore, 
any  significant  change  to  the  postulated  funding  profile — almost  always  a  reduction — is 
usually  disruptive  to  even  the  best-planned  and  most  technically  stable  program.  The 
usual  disruption  involves  the  stretching  out  of  programs  into  the  future,  with  attendant 
delays  in  fielding  of  needed  capabilities,  and  increased  total  costs,  even  if  only  associated 
with  the  lengthened  period  of  incurred  “overhead”  costs. 

Frequent — indeed  chronic — changes  in  the  actual  funding  provided  to  acquisition 
programs  compared  with  the  initially  approved  funding  profiles  is  a  widely  recognized, 
but  not  necessarily  dominant,  cause  of  program  cost  growth.4  On  the  other  hand,  factors 
other  then  funding  instability  tend  to  dominate  the  causes  of  cost  growth  for  the  minority 
of  programs  that  contribute  to  the  majority  of  the  growth  of  the  overall  acquisition  costs 
of  the  Department.  Regardless  of  the  specific  uncertainties  of  the  causes  of  cost  growth 
on  individual  programs,  there  are  actions  the  USD  (AT&L)  could  pursue  in  the  area  of 
better  integration  of  the  acquisition  and  resource  allocation  systems  that  would  help 
stabilize  program  funding.  These  potential  stabilization  actions  are  summarized  below. 
The  first  one  of  these  actions  could  have  a  major  impact. 

1.  Funding  Stability  Issue  1:  Mitigation  of  the  Migration  of  Planned  Acquisition 
Funds  to  the  Operating  Accounts 

The  mismatch  between  the  Department’s  ability  to  carefully  forecast  the  funds 
needed  in  the  future  for  each  approved  weapon  system  and  the  Department’s  ability  (or 
willingness)  to  forecast  its  needs  for  future  operating  funds  is  an  important  contributor  to 
the  instability  of  acquisition  program  funding.  Given  that  DoD  must  always  constrain  its 
total  funding  plans  to  the  level  prescribed  by  the  long-range  budget  plans  of  the  President 
and  Congress,  every  year  the  Department  is  faced  with  the  need  to  cut  back  on  previously 


4  Work  in  progress  at  IDA  indicates  a  need  for  much  better  recording  and  retention  of  program  content 
and  funding  decision  documentation  if  uncertainty  in  the  causes  of  program  cost  growth  are  to  be 
reduced. 
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planned  and  approved  acquisition  spending  in  order  to  meet  unplanned  needs  for 
operating  and  maintenance  (O&M)  funds  and  military  personnel  (MILPERS)  costs.5 

This  source  of  instability  in  acquisition  program  funding  is  chronic — not  just 
associated  with  either  the  current  high  tempo  of  unplanned  operations  or  with  deficit 
concerns.  Indeed,  a  former  Deputy  Secretary  of  Defense  recently  characterized  DOD’s 
long-standing  budgeting  practices  for  O&M  and  MILPERS  as  knowingly  “dishonest.”6 
The  acquisition  program  “cuts”  imposed  as  a  result  of  this  habitual  practice  are  usually 
broadly  spread  across  most  acquisition  programs,  requiring  that  most  programs  be 
replanned  in  detail  and  at  some  increase  in  total  cost.  And  the  cause  of  this  type  of 
instability  is  not  confined  to  actions  in  the  executive  branch  across  the  5 -year  plan.  In 
some  former  years  the  Congress  would  level  a  “tax”  on  DoD  in  the  form  of  an 
undistributed  reduction  for  the  imminent  budget  year  that  would  have  the  same  broad 
destabilizing  effect. 

Because  this  impact  results  from  purely  budgetary  considerations  and  not  from 
changes  in  the  threat,  or  acquisition  program  problems,  or  other  changes  in  a  particular 
program,  it  is  difficult  to  argue  within  DoD  that  any  particular  program,  or  set  of 
programs,  should  be  sacrificed  in  order  to  protect  the  stability  of  the  remainder,  although 
most  agree  that  such  vertical  cuts  would  be  sensible. 

Clearly,  a  broad  solution  to  this  problem  of  annual  transfers  of  previously  planned 
acquisition  funds  to  operating  and  personnel  accounts  would  be  for  the  Department  to 
plan  its  long-term  O&M  spending  to  the  same  “most  likely  cost”  criteria  that  it  tries  to 
apply  to  the  planned  cost  of  its  weapons  systems.  For  this  reason,  it  is  important  that 
acquisition  program  planning  be  more  closely  coordinated  with  the  Department’s  overall 
resource  allocation  and  budgeting  processes.  The  latter  is  particularly  challenging  in  that 
decisions  on  acquisition  programs  are  largely  event  driven — completion  of  development, 
etc. — and  budgets  are  calendar  driven.  It  is  a  hopeful  sign  that  Secretary  Krieg,  in  his 
recent  testimony,  continued  to  emphasize  the  need  to  better  integrate  acquisition  and 
resource  management.  The  Team  believes  that  real  progress  will  occur  in  this  area  only  if 
the  Secretary  himself  demands  it.  The  DAE  has  little  formal  leverage  over  the  budgeting 
of  operating,  maintenance,  and  military  personnel  funds. 


5  See  the  testimony  of  former  Deputy  Secretary  of  Defense  John  Hamre  to  the  Airland  Subcommittee  of 
the  Senate  Armed  Services  Committee  on  15  November  2005. 

6  Ibid. 
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2.  Funding  Stability  Issue  2:  “Fencing”  Is  Not  the  Solution  to  Stabilizing  Funding 

From  time  to  time  the  suggestion  is  made  that,  once  an  acquisition  program  has 
been  thoroughly  reviewed  and  approved  by  the  Defense  Acquisition  Executive,  the 
associated  funding  profile  that  would  lead  to  the  next  major  milestone  should  be 
“fenced,”  i.e.,  exempted  from  further  adjustments,  such  as  those  frequently  made  during 
the  annual  budget  preparation  cycles.  Indeed,  the  recently  completed  Defense  Acquisition 
Performance  Assessment  project  made  just  such  a  recommendation.7 

As  important  as  improved  funding  stability  is  to  the  coherence  and  efficiency  of 
acquisition  program  management,  it  is  not  more  important  than  the  Department’s  ability 
to  respond  agilely  to  changing  threats,  risks,  and  total  funding  availability  as  it  prepares 
budget  proposals.  Under  current  procedures,  fencing  the  funding  for  some  acquisition 
programs  would  have  the  effect  of  further  destabilizing  others,  For  this  reason,  except  for 
occasional  isolated  programs  of  great  strategic  importance,  Secretaries  of  Defense  have 
prudently,  in  the  Team’s  judgment,  been  reluctant  to  mandate  that  specific  levels  of 
funding  be  eannarked  for  specific  acquisition  programs  as  the  Services  annually  update 
their  long-range  plans  in  response  to  his  guidance. 

3.  Funding  Stability  Issue  3:  The  Need  for  Planned  “Reserves” 

The  lack  of  DoD  “Management  Reserves”  is  frequently  cited  as  a  source  of 
program  instability.  Managers  of  civil  projects  ranging  in  complexity  from  building  a 
single-family  home  to  tunneling  under  a  major  city  know  they  cannot  accurately  predict 
the  total  cost  just  by  adding  up  costs  of  each  of  the  initially  planned  steps  known  to  be 
needed.  There  are  always  “unknowns,”  and  even  “unknown  unknowns,”  that  drive  up  the 
final  cost.  Builders  who  promise  a  result  at  a  fixed  price  always  include  some  unallocated 
contingency  reserves  in  their  bids  based  on  their  professional  experience,  or  they  would 
soon  be  bankrupt. 

In  sharp  contrast  to  private  sector  practice,  the  major  segment  of  the  DoD 
acquisition  spending  that  is  devoted  to  research,  development,  test  and  evaluation,  and 
most  of  the  remainder  that  goes  to  procurement  spending,  is  contracted  not  on  a  fixed- 
price  basis,  but  using  cost-type  contracts.  In  principle,  the  Department  could — and 
should — include  prudent  contingency  reserves  in  the  estimates  of  both  development  and 


7  http://www.acq.osd.mil/dapaproject. 
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procurement  costs.  In  practice  however,  there  is  a  bias  against  such  prudent  planning  in 
large  part  because  there  is  always  some  need  that  is  more  tangible  than  the  unknowns  that 
motivate  planning  for  reserves. 

The  use  of  independent  cost  estimates  has  proven  useful  for  establishing  program 
“baseline”  costs  that  are  generally  more  likely  to  be  accurate  than  are  “bottoms  up” 
program  cost  estimates  that  are  devoid  of  reserves.  The  Department  has  long  supported 
an  independent  Cost  Analysis  Improvement  Group  (CAIG)  dedicated  to  improving  its 
estimates  of  future  costs  of  major  weapons  systems.  It  has  also  supported  Service 
independent  cost  estimating  capabilities.  Based  on  IDA’s  previously  cited  2004 
examination  of  data  on  138  programs,  it  appears  that  the  decades-long  effort  to  align 
DoD  acquisition  procurement  budgets  with  the  results  of  truly  independent  cost  estimates 
has  been  modestly  successful  in  reducing  apparent  production  cost  overruns.  That 
analysis  showed  that  about  75%  of  the  cost  growth  in  production  programs  was 
attributable  to  only  about  25%  of  the  programs — outliers  beyond  the  expected  normal 
distribution  of  estimate  errors.  Many  of  these  outliers  are  thought  to  be  attributable  to 
technological  immaturity  at  the  time  the  initial  cost  estimates  were  made,  as  discussed  in 
Section  II  of  this  report,  as  well  as  some  “requirements  creep”  that  may  or  may  not  have 
received  sufficient  oversight. 

Because  of  the  uniqueness  of  every  development  program,  and  the  sparseness  of 
analytic  tools,  independent  estimates  of  development  costs  have  proved  somewhat  less 
reliable  than  those  for  production  costs.  Furthennore,  despite  DoD  policies  that  support 
the  use  of  CAIG  cost  estimates  for  establishing  Acquisition  Program  Baselines  (APB) 
and  Selective  Acquisition  Reports  (SAR)  to  Congress,  the  siren  song  “this  program  is 
different”  has  often  led  to  the  acceptance  of  cost  estimates  that  are  lower  than  those 
provided  by  the  CAIG.  Such  practices  lead  to  the  Department’s  tendency  ,  on  average,  to 
accept  rosy  forecasts  of  development  costs 

The  need  for  planned  reserves  is,  of  course,  a  long-standing  issue.  In  past  years 
the  Department  has  tried  several  different  approaches  to  establishing  prudent  levels  of 
reserves,  at  least  in  its  outyear  plans,  that  can  be  allocated  as  needed  to  salvage  troubled 
programs,  or  accommodate  other  sensible  changes,  without  having  to  tax  and  destabilize 
others.  In  all  cases,  such  schemes  have  been  abandoned  after  only  a  few  years,  or  even 
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months,  because  of  the  difficulty  in  holding  back  funds — even  outyear  funds — against 
unknown  eventualities,  when  there  were  so  many  competing  demands  to  meet  known 
needs. 

The  Team’s  sense  is  that  the  stability  of  acquisition  programs  is  important  enough 
to  warrant  yet  another  try  at  establishing  such  a  reserves  program — not  necessarily  for  the 
budget  year,  but  surely  for  the  outyear  plan.  The  extent  to  which  such  reserves  should  be 
held  at  the  Program  Executive  Officer  (PEO),  Service  Acquisition  Executive  (SAE,)  or 
Defense  Acquisition  Executive  (DAE)  level  has  not  been  addressed  in  this  project 

4.  Funding  Stability  Issue  4:  Emerging,  Unplanned,  Programs 

An  additional  source  of  instability  is  a  decision  to  move  a  promising  experimental 
program  that  had  not  been  planned  for  full-scale  development  and  production  into  the 
formal  acquisition  system.  The  Department  funds  a  variety  of  promising  experiments  that 
each  have  some  prospect  of  becoming  worthy  of  longer-term  funding  than  originally 
planned.  Such  programs  are  typically  managed  by  the  Defense  Advanced  Research 
Projects  Agency  (DARPA)  or  in  the  Department’s  Advanced  Concept  Technology 
Demonstrator  (ACTD)  portfolio.  If  one  or  more  such  programs  are  tested  and  found  to 
warrant  prompt  inclusion  in  the  Department’s  long-term  program  plans,  they  usually 
must  displace  one  or  more  other  programs — a  destabilizing  activity  that  has  a  chilling 
effect  on  the  mainstream  defense  acquisition  community’s  enthusiasm  for  such  programs. 

One  obvious  solution  to  this  threat  would  be  for  the  Department  to  create  a 
standing  outyear  “wedge”  of  unallocated  funds,  some  of  which  could  be  shifted  annually 
to  fund  emerging  good  ideas  without  disrupting  other  programs.  A  problem  with  this 
approach  is  similar  to  that  for  establishing  an  outyear  reserve  for  funding  troubled 
programs  and  other  program  changes — there  would  be  a  significant  one-time 
destabilization  of  other  programs  in  order  to  establish  such  contingency  accounts. 

The  Team  judges  that  the  problems  that  would  be  entailed  in  attempting  such  a 
large  one-time  reallocation  of  planned  funding  outweigh  the  problems  entailed  in 
continuing  the  current  case-by-case  decision  process.  Stability  could  still  be  increased, 
however,  if  the  necessary  offsets  were  to  be  identified  as  vertical  cuts  of  specific 
programs  rather  than  as  broad  taxes  across  many  programs. 
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5.  Funding  Stability  Issue  5:  Milestone  Budgeting 

A  different  step  toward  stabilizing  the  funding  for  the  development/initial 
production  phase  of  programs  would  be  to  shift  to  “milestone  budgeting”  as 
recommended  by  the  Packard  Commission.  Under  this  approach  the  full  estimated  cost  of 
development,  and  perhaps  the  first  year  or  two  of  initial  production,  would  be 
appropriated  and  managed  as  a  lump  sum  in  much  the  same  way  that  the  cost  of  a  new 
lead  ship  used  to  be  appropriated  and  managed.  A  full-scale  change  to  such  a  procedure 
would  have  a  major  impact  only  on  obligational  authority,  but  not  on  actual  outlay  rates, 
as  the  funds  would  be  actually  expended  at  approximately  the  previously  planned  rate. 
The  benefit  would  be  the  greatly  increased  ability  of  program  managers  to  efficiently 
plan  and  execute  the  multiyear  activities  of  their  development  programs  due  to  the 
confidence  they  would  have  in  the  availability  of  funds. 

Despite  the  attractive  features  of  this  concept,  the  Team  believes  that,  as  with 
establishing  a  wedge  for  new  ideas,  the  one-time  apparent  cost  of  shifting  all  acquisition 
programs  to  milestone  budgeting,  including  the  likely  difficulties  associated  with 
perceived  reductions  in  congressional  oversight,  outweigh  the  likely  benefits  of  devoting 
a  major  near-tenn  effort  to  this  potential  management  initiative.  However,  one  or  more 
pilot  projects  of  this  type  could  be  pursued  to  demonstrate  the  benefits. 

C.  THE  INTEGRATION  OF  DOD  PROCESSES:  BETTER  JOINT  PROGRAM 
AND  ACQUISITION  PLANNING 

The  impediments  to  jointness  in  the  Department  of  Defense  acquisition  program 
planning  are  well  known.  Although  the  Goldwater-Nichols  Act  is  widely  acknowledged 
to  have  led  to  significant  improvements  in  the  planning  and  execution  of  joint  military 
operations,  its  impact  on  increasing  the  jointness  of  the  Department’s  long-range 
program  planning  process,  including  its  acquisition  program  planning,  has  been  much 
less  impressive  to  date. 

It  may  be  overly  simplistic  to  contrast  the  apparent  “internal”  jointness  between 
the  air  and  ground  elements  of  the  US  Marine  Corps  with  the  ongoing  difficulties  in 
achieving  similar  synergies  between  the  Army  and  the  Air  Force,  but  there  are  two 
important  features  of  successful  joint  acquisition  projects  that  can  be  illustrated  using  that 
analogy:  (1)  joint  operating  concepts,  and  (2)  joint  funding. 
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1.  Joint  Operating  Concepts 


First,  a  common,  unified  concept  of  how  to  operate  together  not  only  improves 
field  performance,  but  is  essential  to  real  jointness  in  acquisition.  The  Marines  pride 
themselves  on  having  established  effective  operational  concepts  for  air-ground  operations 
across  a  broad  range  of  combat  scenarios.  The  unity  of  such  concepts  is  widely  advertised 
as  integral  to  their  “MAGTF”  organizational  structures.  However,  joint  operating 
concepts  and  joint  integrating  concepts  that  cross  the  boundaries  of  the  other  Services  are 
much  less  well  developed. 

It  is  difficult  to  make  a  lot  of  progress  in  defining  an  acceptable  set  of  joint 
requirements  for  new  equipment  that  would  be  operated  by  multiple  Services  (or  operated 
in  a  mutually  dependent  fashion  among  two  or  more  Services)  in  the  absence  of  agreed, 
and  relatively  specific,  joint  operating  concepts.  The  Joint  Staff  is  working  to  develop  a 
broad  range  of  joint  operating  and  integrating  concepts  to  address  this  need,  but  progress 
is  glacial.  This,  plus  the  joint  funding  issue  discussed  below,  constitutes  the  major 
impediment  to  achieving  the  long-standing  goal  of  having  acquisition  programs  “born 
joint.” 

The  closest  the  Department  has  come  in  recent  years  to  a  successful  major  joint 
acquisition  program  is  the  Joint  Strike  Fighter — now  the  F-35.  Research  at  IDA  into  the 
differences  between  the  originally  joint  F-  111  program  that  reverted  to  a  single  Service, 
and  the  largely  successful  (to  date)  joint  F-35  program,  identified  one  overriding  factor.8 
In  the  F-l  1 1  program  the  Navy  and  Air  Force  failed  to  agree  on  an  acceptable  set  of  joint 
performance  requirements,  largely  because  of  differing  operating  concepts.  Such 
agreement  on  performance  requirements,  reached  early  in  the  F-35  program,  has  largely 
persisted  through  many  changes,  by  virtue  of  the  joint  management  and  funding  structure. 

But  the  F-35  type  of  “jointness”  flows  more  from  a  desire  to  save  acquisition  and 
maintenance  costs  through  the  use  of  common  equipment  than  it  does  from  the  need  to 
operate  more  jointly.  Even  with  common  airframes,  the  Services  could  in  principle  equip 
them  with  uncommon  sensors,  communications  equipment,  and  weapons — the  type  of 
equipment  more  important  to  true  joint  operations  than  is  airframe  commonality. 


David  McNicol  et  al.  The  Accuracy  of  Independent  Estimates  of  the  Procurement  Costs  of  Major 
Systems,  IDA  Paper  P-3989,  August  2005. 
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The  lack  of  planned  interoperability  among  the  military  services  is  not  a  new 
problem  even  though  warfighters  in  the  field  have  a  recent  record  of  moderately 
successful  last-minute  improvisation,  that  ends  up  getting  the  job  done,  albeit  at 
considerable  expense  in  time  and  efficiency.  In  discussing  the  acquisition  of  equipment 
important  to  the  interoperability  of  US  forces,  the  2004  report  Beyond  Goldwater-Nichols 
(BG-N)  from  the  Center  for  Strategic  and  International  Studies  stated: 

This  enduring  lack  of  jointness  in  how  DoD  procures  weapons  has  both 
raised  the  cost  of  military  operations  (e.g.,  persistent  interoperability 
problems  cause  friendly  fire  casualties)  and  constrained  the  growth  of  US 
military  capabilities  (e.g.,  Services  invest  too  much  in  duplicative 
capabilities  and  too  little  in  Low  Density/  High  Demand  assets). 

Nowhere  is  the  need  for  improved  coherence  in  the  acquisition  of  military 
capabilities  more  apparent  than  in  command,  control,  and  communications  systems. 
Citing  “repeated  failures  over  the  past  decade  to  develop  common,  interoperable” 
command  and  control  systems,  the  aforementioned  CSIS  report  explicitly  recommended 
that  funding  and  responsibility  for  managing  such  programs  be  transferred  from  the 
Services  to  a  new  joint  management  entity. 

The  Department  is  looking  broadly  into  the  planning  for  such  joint  command  and 
control  and  supporting  information  infrastructure  programs.  The  central  issue  being 
addressed  is  reportedly  how  to  assure  that  separately  acquired  and  fielded  programs 
provide  the  necessary  integrated  joint  capabilities,  and  not  the  joint  acquisition  and 
management  approach  recommended  by  CSIS  and  others. 

Although  the  Under  Secretary  for  Acquisition,  Technology  and  Logistics  has  both 
the  responsibility  and  authority  to  decide  what  the  Department  will  propose  to  buy,  he  is 
not  chartered  or  staffed  to  establish  joint  operating  concepts.  The  absence  of  such  joint 
concepts,  which  can  be  particularly  difficult  when  they  open  up  Inter-Service  Title  10 
roles  and  missions  issues,  too  often  become  apparent  only  very  late  in  the  process  of 
establishing  new  Service-unique  acquisition  programs.  A  recent  example  involved  the 
DAE  exercising  his  authority  by  delaying  an  Army  plan  to  issue  an  RFP  for  a  new 
tactical  airlift  plane  in  order  to  provide  an  opportunity  to  establish  a  joint  Army/Air  Force 
program  for  acquiring  a  common  aircraft. 

The  Team  was  encouraged  to  read  Secretary  Krieg’s  testimony  of 
27  September  2005  in  which  he  reported  that  he,  too,  believes  the  Department’s 
requirements  and  acquisition  processes  must  be  better  integrated,  and  laid  out  his 
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principles  for  achieving  that  end.  He  has  reportedly  moved  to  take  a  greater  role  in  the 
examination  of  requirements,  not  only  for  major  defense  acquisition  programs  (MDAPs) 
but  also  for  those  other  programs  that  may  not  meet  the  MDAP  cost  thresholds  but  are 
vital  enablers  to  future  joint  military  operations,  such  as  command  and  control  systems. 
This  latter  consideration  reinforces  the  Team’s  belief  that  a  greater  degree  of  unity  could 
be  achieved  in  DoD’s  acquisition  management  practices  if  the  command  and  control- 
related  acquisition  programs  now  structured  and  overseen  by  the  Assistant  Secretary  of 
Defense,  for  Networks  and  Information  Integration  [ASD  (Nil)];  the  Commander,  Joint 
Forces  Command;  and  the  Joint  Staff  (J-6)  were  more  clearly  brought  within  the  purview 
and  processes  of  the  USD(AT&L). 

Similarly,  the  Team  notes  the  lack  of  a  management  mechanism  for  examining 
the  prospects  for  “joint”  Special  Acquisition  Programs  (SAPs).  At  present  it  appears  that 
the  Services  each  establish  and  manage  their  own  SAPs,  and  control  access  thereto,  in 
ways  that  largely  preclude  consideration  of  joint  applications.  As  with  the  acquisition 
programs  now  overseen  by  Nil,  the  Department’s  Special  Access  Programs  would  likely 
benefit  from  greater  oversight  by  the  USD(AT&L),  with  early  attention  given  to  potential 
joint  applications. 

2.  Joint  Funding 

Second,  a  single  flow  path  for  funding  clearly  helps.  When  the  Marines  plan  their 
future  spending  programs,  they  can  internally  resolve  issues  and  assemble,  at  least  within 
the  Department  of  the  Navy,  a  coherent  long-range  program  plan  for  the  several  systems 
important  to  effective  joint  air-ground  operations.  Counterexamples  include  the 
integrated  radar  and  missile  air  defense  systems  that  were  developed  separately  by  the 
Army,  Navy,  and  Air  Force,  which  work  well  within,  but  not  across,  Service  boundaries. 
There  are  no  examples  of  single  sources  of  funding  for  successful  major  cross-Service 
weapons  acquisition  programs  that  have  been  fielded.  The  Missile  Defense  Agency 
(MDA)  has  a  single  source  of  funding  only  for  the  development  of  ballistic  missile 
defenses,  not  their  production,  fielding,  and  support.  One  could  argue,  moreover  ,  that  the 
recent  success  of  the  Navy  Aegis  Ship  in  intercepting  test  vehicles  was  a  direct  result  of 
MDA  “joint”  development  funding,  and  would  not  likely  have  been  directly  funded  by 
the  Navy. 

For  the  most  part,  “purple”  or  “joint”  funding  of  common  support  activities  has 
proven  its  worth.  Despite  their  considerable  birthing  and  growing  pains,  such  DoD-wide 
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activities  as  the  Defense  Logistics  Agency,  Defense  Information  Services  Agency, 
Defense  Finance  and  Accounting  Service,  and  the  Defense  Contract  Management 
Agency,  are  widely  agreed  to  now  be  working  well  and  are  considerably  less  costly  than 
would  have  been  the  case  had  the  Services  each  retained  such  functions.  Additional 
implementation  of  such  “joint”  funding  appears  to  be  a  necessary  condition  for  achieving 
much  real  progress  towards  joint  acquisition. 

In  fact,  it  may  take  another  Herculean  effort,  such  as  that  that  went  into  the 
Goldwater-Nichols  Act  itself,  to  boldly  move  the  Department  into  a  new  approach  to 
acquiring  capabilities  that  are  truly  “born  joint.”  It  seems  unrealistic  to  expect  much  more 
progress  toward  improved  joint  acquisition  without  a  major  effort  by  both  the  Secretary 
of  Defense  and  the  Congress. 

D.  RECOMMENDATIONS  FOR  IMPROVING  PROGRAM  STABILITY  AND 
JOINTNESS 

The  foregoing  assessments  suggest  that  the  following  three  “requirements”  and 
“funding  stability”  items  warrant  the  Under  Secretary’s  consideration  for  renewed  near- 
term  emphasis: 

1.  The  establishment  of  a  more  robust  process  for  identifying  and  analyzing 
broad  alternatives  for  meeting  needed  capabilities  well  before  the  Defense 
Acquisition  Board  is  asked  to  review,  and  the  DAE  must  decide  on,  specific 
characteristics  of  a  new  weapon  system  and  a  schedule  for  it,.  The 
USD(AT&L)  would  be  a  co-equal  with  PA&E  in  this  strengthened  AoA 
process.  This  process  would  precede  but  be  closely  associated  with  a 
revitalized  “Milestone  A”  process  that  is  intended  to  elevate  attention  to 
needed  jointness  considerations,  technology  maturation,  and  systems 
engineering  practices  prior  to  formal  Program  Initiation  at  Milestone  B,  as 
described  below  and  in  section  II  of  this  report.  Getting  acquisition  programs 
started  on  a  sound  technical  and  financial  footing  is  much  better  than  having 
to  salvage  and  restructure  troubled  programs  later. 

2.  A  proposal  to  the  Secretary  and  the  Under  Secretary  of  Defense  for  Personnel 
and  Readiness  [USD(P&R)j  and  the  Under  Secretary  of  Defense 
(Comptroller)  [USD(C)]  for  improved  accuracy  in  the  forward  planning  of 
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O&M  and  MILPERS  costs  to  be  overseen  by  the  Secretary’s  principal  staff 
assistants  in  those  areas. 

3.  Reestablishment  of  a  process  for  programming  suitable  levels  of  reserves  for 
acquisition  programs  in  the  years  beyond  the  budget  year.  The  appropriate 
holder  of  such  reserves  (DAE/S AE/PEO)  would  be  subject  to  additional  near- 
term  study. 

4.  To  promote  and  facilitate  joint  acquisition  activities,  the  USD(AT&L)  could 
realign  his  staff  such  that  one  segment  would  have  a  primary  cross-mission 
responsibility  to  seek  out  emerging  Service  acquisition  programs  that  have 
potential  for  joint  acquisition  such  as  at  the  revitalized  Milestone  “A”  point 
discussed  in  the  preceding  section — well  before  they  are  formally  brought  to 
the  Defense  Acquisition  Board  (DAB)  for  Milestone  “B”  Review. 

5.  The  Under  Secretary  should  more  routinely  exercise  his  authority  over  those 
acquisition  management  activities  particularly  important  to  joint  warfighting 
(including  “requirements”  determination)  that  have  migrated  to  other  offices, 
particularly  CIO,  Nil,  and  Service  SAP  offices.  This  measure  would  help 
restore  unity  to  acquisition  management  and  oversight. 
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II.  IMPROVED  OUTCOMES  THROUGH  BETTER 
SYSTEMS  MANAGEMENT  PRACTICE 


A  major  thread  of  the  Packard  Commission’s  recommendations  was  to  “fly  before 
buy.”  Secretary  Krieg  now  calls  it  “try  before  buy.”  However  it  is  stated,  this  concept  remains 
as  valid  today  as  it  was  almost  20  years  ago.  In  fact,  it  could  be  argued  that  it  is  even  more 
valid  today  because  there  is  no  major  new  threat  so  imminent  that  sound  system  engineering 
management  practices  need  to  be  sacrificed  in  order  to  accelerate  the  fielding  of  unproven 
major  equipment.  Although  the  importance  of  systems  management  is  embedded  in  DoD 
policies,  these  policies  have  been  insufficiently  heeded  in  recent  years. 

The  Department  has  relied  heavily  on  three  key  tools  that  are  intended  to  implement 
this  principle:  One  is  working  well  (operational  testing);  the  other  two  are  not  (systems 
engineering  and  technology  readiness  assessments),  as  discussed  below. 

A.  FORMAL  OPERATIONAL  TESTING 

Significant  benefits  have  accrued  to  our  warfighters  by  independently  ensuring  that 
their  equipment  has  demonstrated  both  operational  effectiveness  and  suitability  in  the  field. 
US  military  equipment  is  the  envy  of  the  world’s  fighting  forces,  in  large  part  as  a  result  of 
rigorous  and  independent  testing.  The  integrity  of  this  process  was  supported  by  establishing  a 
direct  reporting  relationship  between  the  Director  of  Operational  Testing  and  the  Secretary  of 
Defense.  In  addition  to  verifying  the  readiness  of  a  particular  system  to  be  fielded,  the  data 
collected  during  the  operational  testing  of  many  systems  provides  insights  into  problematic 
trends  in  DoD’s  systems  engineering  practices,  as  discussed  below. 

One  complaint  about  the  current  operational  testing  process  is  that  the  testing 
community  is  sometimes  seen  as  testing  systems  against  obsolete  “requirements”.  In  this 
regard,  as  the  USD(AT&L)  increases  his  involvement  in  overseeing  the  requirements  process 
as  discussed  in  Section  I  of  this  report,  a  productive  avenue  of  activity  would  be  to  establish  a 
process  that  better  ensures  that  fonnal  requirements  documents  and  associated  “test  plans”  are 
kept  current.  Although  important,  the  Team  does  not  believe  this  recommendation  merits  a 
major  initiative  at  the  Defense  Acquisition  Executive  level. 


B.  ADEQUATE  SYSTEMS  ENGINEERING  IN  PROGRAMS 


Good  systems  engineering  is  critical  to  the  successful  development  and  production  of 
complex  weapons  systems.  Things  will  inevitably  go  wrong  when  an  event-driven  systems 
engineering  process  is  improperly  dominated  by  a  schedule-driven  systems  integration 
environment.  Cost-perfonnance  trades  are  being  made  too  late,  and  immature  technology  is 
being  relied  upon  too  often.  As  a  result,  rework  ensues,  schedules  slip,  and  cost  grows. 

The  consequences  of  poor  systems  engineering  discipline  have  been  well  documented. 
The  particular  problems  associated  with  the  risk  of  proceeding  into  full-scale  development 
with  immature  but  critical  technology  will  be  treated  in  a  subsequent  subsection.  The  general 
problem  is  illustrated  by  the  results  of  a  recent  Government  Accountability  Office  (GAO) 
report  that  found  that  at  their  point  of  critical  design  review  (CDR),  only  42%  of  DoD’s 
programs  demonstrated  adequate  design  stability. 1 

•  Development  programs  with  stable  designs  at  CDR  averaged  6%  subsequent  cost 
growth. 

•  Development  programs  without  stable  designs  at  CDR  averaged  46%  subsequent 
cost  growth  and  a  29-month  schedule  delay. 

While  DoD  has  not  confirmed  these  specific  estimates,  there  is  little  doubt  as  to  the 
general  accuracy  of  implication:  unstable  designs  at  CDR  presage  large  growth  in  costs. 

The  consequences  of  weak  systems  engineering  not  only  involve  short  term  cost 
growth  and  schedule  slips,  but  extend  to  increased  operating  and  support  costs  after  weapons 
systems  are  fielded.  Operational  testing  prior  to  full-rate  production  is  intended  to 
demonstrate  the  achievement  of  the  two  most  important  aspects  of  weapon  system 
performance:  (1)  nominal  combat  effectiveness  when  employed  as  envisioned,  and 
(2)  battlefield  suitability  in  terms  of  reliability  and  supportability  etc.  Both  attributes  are  of 
comparable  importance  in  judging  the  success  of  an  acquisition  program. 

In  recent  years  there  has  been  a  substantial  drop  in  the  percentage  of  systems  passing 
suitability  testing,  while  at  the  same  time  nominal  combat  effectiveness  passing  rates  have 
increased.  These  trends  are  shown  in  Figure  II- 1.2 


1  Defense  Acquisitions:  Assessments  of  Selected  Major  Weapon  Programs,  GAO-05-301,  March  2005. 

-  There  has  been  a  trend  of  more  systems  passing  operational  test  from  an  effectiveness  perspective.  This 
trend  is  primarily  the  result  of  a  change  in  testing  philosophy.  Early  on,  tests  were  mostly  conducted  on  a 
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Figure  11-1.  Percentage  of  Systems  Passing  Operational  Test 


Declining  suitability  numbers  and  their  divergence  from  the  effectiveness  numbers 
clearly  demonstrate  that  both  government  and  contractor  program  offices  are  shifting 
emphasis  away  from  the  suitability  goals  whose  achievement  is  highly  dependent  on  sound 
systems  engineering  practices.  Resources  needed  to  meet  suitability  goals  (which  themselves 
have  been  deemphasized  as  testable  contractual  obligations)  are  often  diverted  to  deal  with 
technical  performance  problems  with  the  hope  that  suitability  can  be  improved  after 
production  starts — usually  at  increased  cost.  The  drop  in  suitability  shown  in  Figure  II- 1  can 
also  be  attributed  to  a  decline  in  systems  engineering  expertise  in  program  offices  based  on  an 
overreaction  to  “acquisition  reform”  initiatives  as  described  later  in  this  section. 

A  systems  engineering  problem  of  comparable  importance  to  that  of  declining 
suitability  is  the  weak  application  of  current  rules  for  ensuring  that  technical  risk  has  been 
adequately  limited,  as  discussed  in  section  II. C,  below. 

Having  perceived  problems  with  systems  engineering  in  major  DoD  acquisition 
programs,  Under  Secretary  Edward  C.  “Pete”  Aldridge  Jr.* * 3  established  a  goal  to  drive  good 
systems  engineering  practice  back  into  the  way  DoD  does  business.  To  this  end,  in  late  2002 
he  established  an  office  in  OUSD(AT&L)  dedicated  to  spearheading  these  efforts. 

A  number  of  parallel  studies  have  been  conducted  to  identify  the  specific  system 
engineering  concerns  that  needed  to  be  addressed.  In  January  2003,  a  National  Defense 


pass-fail  basis  against  a  specific  (and  sometimes  arbitrary)  number.  In  today’s  environment,  testing  is  based 

on  the  ability  to  accomplish  the  mission. 

3  The  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics  from8  May  2001  to 
23  May  2003. 


II-3 


Industry  Association  (NDIA)  study  reported  the  following  issues  from  both  government  and 
contractor  perspectives: 

•  Lack  of  awareness  of  the  importance,  value,  timing,  accountability,  and 
organizational  structure  of  systems  engineering  on  programs 

•  Unavailability  of  adequate,  qualified  resources  within  government  and  industry  for 
allocation  on  major  programs 

•  Insufficient  systems  engineering  tools  and  environments  to  effectively  execute 
systems  engineering  on  programs 

•  Poor  initial  program  formulation 

•  Requirements  definition,  development,  and  management  is  not  applied 
consistently  and  effectively 

A  2004  DoD-directed  study  cited  similar  issues  as  root  causes  of  failures  in  acquisition 
projects:4 

•  Inadequate  understanding  of  requirements 

•  Lack  of  systems  engineering  discipline,  authority,  and  resources 

•  Lack  of  technical  planning  and  oversight 

•  Stovepipe  developments  with  late  integration 

•  Lack  of  subject  matter  expertise  at  integration  level 

All  of  these  issues  are  predominantly  driven  by  the  poor  application  of  sound  systems 
engineering  practices  in  acquisition  programs. 

To  confirm  the  foregoing  observations,  a  recent  NDIA  survey  asked  industry 
members,  “How  would  you  rate  the  quality  and/or  effectiveness  of  the  [government] 
customer’s  program  manager  systems  engineering  capability?”  As  reported  in  a  May  2005 
Systems  Engineering  Town  Hall,  the  synthesized  answer  was:  “Presently,  even  within  a  given 
Service,  this  varies  significantly  from  program  to  program.  There  is  a  decided  lack  of  systems 
engineering  capability  and  understanding  compared  with  pre-acquisition  reform  days." 
(emphasis  added) 


4  As  reported  in  the  DoD  Systems  Engineering  “Town  Hall,”  18  May  2005. 
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This  survey  response  helps  explain  the  data  divergence  shown  in  Figure  II- 1  and 
confirms  a  significant  decline  in  the  Department’s  attention  to  sound  systems  engineering 
practices.  Still,  the  Defense  Acquisition  University  did  not  stop  teaching  good  systems 
engineering  practices  as  a  result  of  Acquisition  Reform.  Similarly,  competent  systems 
engineers  in  the  workforce  did  not  all  retire  or  forget  how  to  do  the  job;  there  are  literally 
thousands  of  them  left  (albeit  fewer  of  them  because  of  recent  hiring  freezes). 

The  Team  believes  that  the  observed  deterioration  in  DoD  systems  engineering 
discipline  is  an  unintended  consequence  of  two  key  tenets  of  the  Acquisition  Refonn  efforts 
of  the  1990s: 

•  Increased  reliance  on  the  defense  industry  itself  to  oversee  complex  acquisition 
programs 

•  Reduced  government  oversight  activities  that  industry  believed  led  to  increased 
cost  and  schedule  slippages 

It  appears  that  in  many  cases  the  Department’s  response  to  these  reform  initiatives 
went  too  far  both  from  a  technical  perspective  and  from  a  program  management  perspective. 
Not  only  did  the  Department  nearly  eliminate  its  involvement  in  systems  engineering 
technical  details  by  assigning  many  fewer  systems  engineers  to  each  program  office,  the 
Department  also  severely  curtailed  the  extent  of  its  systems  engineering  oversight.  This  effect 
was  compounded  by  refonn-driven  reorganizations  in  OUSD(AT&L)  that  deemphasized  the 
systems  engineering  policy  function — reorganizations  that  were  also  mirrored  in  the  Services. 

As  part  of  the  attempt  to  reduce  oversight  practices  that  industry  found  intrusive,  many 
new  development  contracts  eschewed  the  traditional  requirements  for  the  types  of  formal 
reliability  and  supportability  pre-production  testing  that  previously  resulted  from  more 
thorough  government  systems  engineering  oversight.  The  resulting  lack  of  contractual 
requirements  for  testable  “suitability”  measures  appears  to  have  contributed  directly  to  the 
deterioration  of  “suitability”  in  operational  testing  as  shown  in  Figure  II- 1 

Having  observed  some  of  the  deleterious  effects  of  previous  changes,  the  USD 
(AT&L)  Systems  Engineering  Office  promulgated  robust  new  policy  and  guidance  intended 
to  correct  these  problems,5  but  recent  program  assessments  indicate  that  the  problems  persist. 


5  DoD  Directive  5000.1,  The  Defense  Acquisition  System,  12  May  2003;  DoD  Instruction  5000.2,  Operation 
of  the  Defense  Acquisition  System,  12,  May  2003;  The  Defense  Acquisition  Guidebook, 
http://akss.dau.mil/DAG/;  USD(AT&L)  Memorandum  Policy  for  Systems  Engineering  in  DoD, 
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For  example,  the  systems  engineering  plans  long  required  at  major  acquisition  milestones 
have  been  consistently  inadequate.  While  program  managers  probably  understand  that 
placing  resources  and  management  attention  on  systems  engineering  will  be  good  for  the 
program  in  the  long  run,  short-term  issues  continue  to  receive  the  greater  priority  in  many 
instances. 

If  all  of  the  systems  engineering  management  conditions  for  a  successful  program 
were  to  be  met  before  formal  program  initiation  at  Milestone  B,  program  managers  would  be 
in  a  much  better  position  to  maintain  cost  and  schedule.  This  contrasts  sharply  with  the 
current  situation  in  which  systems  engineering  plans  are  hastily  drafted  just  prior  to  Milestone 
B  and  therefore  do  not  inform  or  in  some  cases  even  reflect  the  important  multiyear  risk 
reduction  efforts  that  should  have  been  started  at  Milestone  A  in  order  to  achieve  acceptable 
levels  of  technical  risk  by  Milestone  B.  Under  current  practice,  real  oversight  of  MDAPs  by 
the  USD(AT&L)  doesn’t  currently  begin  until  Milestone  B  is  imminent.  Greater  scrutiny 
starting  at  Milestone  A,  combined  with  more  strictly  enforced  technical  maturity  and  systems 
engineering  criteria  for  formal  program  initiation,  should  lead  to  significantly  less  scrutiny 
later  in  the  acquisition  cycle.  The  net  result  would  be  an  overall  reduction  in  OSD  oversight 
and  a  shift  of  staff  effort  away  from  trying  to  help  salvage  troubled  programs  and  toward 
better  early  program  planning  and  execution. 

C.  TECHNICAL  READINESS  ASSESSMENTS 

Unambiguous  DoD  policy  and  guidance  requires  that  certain  levels  of  technology 
maturity  be  demonstrated  before  formal  acquisition  program  initiation  at  Milestone  B.  It  is 
widely  acknowledged  that  this  guidance  is  too  often  being  waived. 

DoD  Instruction  5000.1,  which  addresses  the  operation  of  the  Defense  Acquisition 
System,  established  a  requirement  to  perform  a  technology  readiness  assessment  (TRA)  prior 
to  formal  program  initiation  and  set  an  entrance  criterion  that  the  “technology  . . .  shall  have 
been  demonstrated  in  a  relevant  environment  or,  preferably  in  an  operational  environment,  to 
be  considered  mature  enough  to  use  for  product  development  in  systems  integration  ...  if  the 
technology  is  not  mature,  the  DoD  Component  shall  use  alternative  technology  that  is  mature 
and  that  can  meet  the  user’s  needs.” 


20  February  2004;  USD(AT&L)  Memorandum  Policy  Addendum  for  Systems  Engineering, 
22  October  2004. 
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As  the  Congress  recognized  when  it  passed  Section  804  of  the  National  Defense 
Authorization  Act  of  Fiscal  Year  2002,  and  as  the  GAO  noted  in  its  recent  reports,  the 
Department  and  its  component  Services  have  too  often  departed  from  these  well-established 
rules.  Programs  continue  to  suffer  cost  and  schedule  growth  as  a  result  of  proceeding  into 
system  development  before  key  technologies  are  sufficiently  mature. 

One  reason  why  acquisition  executives  frequently  waive  these  clear  requirements  is 
that,  under  current  DoD  oversight  practices,  the  specific  technical  risks  faced  by  emerging 
programs  have  very  little  visibility  prior  to  being  considered  at  the  point  of  formal  program 
initiation  -  Milestone  “B.”  At  that  point  program  schedules  have  been  established  and  outyear 
funding  laid  in  such  that  Milestone  B  disapproval  because  of  technical  immaturity  would 
seriously  disrupt  the  Service’s  plans,  including  the  award  of  the  system  development  and 
demonstration  contract.  A  more  fonnal  requirement  for  early  identification  and  review  of 
technical  risks  starting  at  Milestone  “A”  would  significantly  ameliorate  this  problem. 

Despite  DoD  policy  that  reflects  a  preference  for  evolutionary  acquisition,  the  Service 
Chiefs  frequently  bring  forward  “requirements”  for  new  weapons  systems  whose  performance 
goals  require  technologies  that  are  immature  at  best.  In  recent  years  several  such  programs 
have  been  approved  for  Milestone  B  initiation  despite  the  fact  that  many  of  the  critical 
technologies  needed  to  achieve  such  capability  were  years  away  from  adequate  maturity,  and 
mature  alternatives  were  not  available. 

The  GAO  has  estimated  the  effect  of  such  departures  from  policy  in  an  analysis  of  54 
major  programs.6  Only  15%  of  these  programs  began  full  systems  development  with  mature 
technology: 

•  Programs  that  started  with  mature  technologies  averaged  9%  total  development 
cost  growth  and  a  7-month  schedule  delay 

•  Programs  that  did  not  have  mature  technologies  averaged  41%  total  development 
cost  growth  and  a  13-month  schedule  delay 

Again,  while  DoD  has  not  confirmed  the  accuracy  of  GAO’s  specific  estimates,  there 
is  no  doubt  about  the  accuracy  of  the  implications.  The  Army’s  Future  Combat  System  (FCS) 
is  a  well-known  specific  example.  The  2004  IDA  review  of  the  Army’s  FCS  program  noted  a 
large  number  of  technical  issues  that  had  not  been  resolved  at  the  time  approval  was  given  to 


6  Defense  Acquisitions:  Assessments  of  Selected  Major  Weapon  Programs,  GAO-05-301,  March  2005. 
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proceed  into  system  design  and  development.  One  result  of  these  unresolved  technical  issues 
was  the  slippage  of  the  Preliminary  Design  Review  by  at  least  1  year  and  overall  slippage  of 
initial  operational  capability  by  4  years.  These  changes  obviously  call  into  question  the 
appropriateness  of  the  Milestone  B  approval  decision  in  2003. 

In  addition  to  the  late  involvement  of  the  Milestone  Decision  Authorities  in  the 
planning  of  key  features  of  new  programs,  including  technology,  the  ability  of  DoD  to 
competently  identify  critical  technologies,  program  funds  to  reduce  their  risk,  and  then  assess 
their  maturity  at  Milestone  B  is  hobbled  by  the  lack  of  independent  technical  expertise.  For 
most  systems  today,  the  program  manager’s  office  itself  identifies  the  technologies  to  be 
assessed.  This  has  caused  some  critical  technology  elements  to  be  overlooked.  In  other 
situations,  the  panel  that  estimates  the  technology  readiness  level  has  had  strong  ties  to  the 
program  office,  which  is  in  turn  strongly  incentivized  to  keep  the  program  on  the  “approved” 
schedule.  This  lack  of  independence  frequently  leads  to  overly  optimistic  assessments  and 
erroneous  predictions  of  future  maturation. 

D.  RECOMMENDATIONS  FOR  IMPROVING  SYSTEMS  MANAGEMENT 

1.  Recommended  Actions  to  Strengthen  the  Department’s  Systems  Engineering 
Management  Practices 

In  conjunction  with  the  revitalized  Analysis  of  Alternatives  (AoA)  process 
recommended  in  section  1,  reinstitute  Milestone  Decision  Authority  (MDA)  oversight  of  the 
Milestone  A  process  so  that  a  sound  systems  engineering  management  plan  is  put  in  place  for 
each  alternative  that  will  be  pursued  into  a  fonnal  risk  reduction/demonstration/validation 
phase  leading  to  a  possible  Milestone  B  formal  initiation  of  a  new  acquisition  program. 

There  should  be  a  very  high  bar  for  waivers  of  an  inadequate  systems  engineering  plan 
at  formal  program  initiation  at  Milestone  B.  OSD  scrutiny  of  programs  should  be  much  more 
intense  for  the  events  leading  up  to  program  initiation.  Program  managers  must  be  held  more 
accountable  for  ensuring  good  systems  engineering  practices  as  discussed  elsewhere  in  this 
report.  Specifics  from  the  programs  systems  engineering  plan  should  be  incorporated  into  the 
annual  operating  plan. 

Industry  must  be  better  incentivized.  Most  importantly,  appropriate  suitability 
demonstrations  should  be  made  co-equal  with  other  key  systems  performance  requirements  in 
the  prime  contracts. 
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2.  Recommended  Actions  to  Strengthen  the  TRA  Process 


•  Require  that  the  formal  identification  of  key  technical  risks  be  made  at  Milestone 
A,  with  specific  funded  plans  to  be  put  in  place  to  achieve  their  specified  level  of 
technical  maturity  by  Milestone  B. 

•  Require  that  sound,  independent  TRAs  be  available  to  all  levels  of  acquisition 
management  well  in  advance  of  Milestone  B. 

•  Raise  the  bar  for  waivers  to  existing  TRA  maturity  levels  such  that  only  programs 
of  great  urgency  and  strategic  importance,  as  agreed  to  by  the  MDA,  are  allowed 
to  proceed  into  SDD  without  demonstrating  the  specified  levels  of  technical 
maturity. 

•  Program  and  budget  sufficient  science  and  technology  funding  to  conduct  the 
necessary  technology  maturity  demonstrations  as  identified  in  each  program’s 
baseline  documentation. 

•  Strengthen  the  technical  competence  and  independence  from  conflict  of  interest  of 
the  government  teams  that 

o  Identify  areas  of  technical  risk,  and 

o  Formally  assess  the  maturity  of  the  associated  technology  developments.7 

This  would  better  assure  both  the  Milestone  Decision  Authorities  and  the  Congress  at  key 
decision  points. 


7  This  could  be  done  by  adapting  the  operational  test  model  to  TRAs.  One  reason  that  operational  testing  is 
so  successful  is  the  independence  and  technical  competence  of  the  Department’s  operational  test  and 
evaluation  staff. 
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III.  IMPROVING  ACCOUNTABILITY  IN  THE 
ACQUISITION  WORKFORCE 


In  its  simplest  terms,  the  Packard  Commission  recommended  that  DoD  implement 
an  industry  model  with  six  attributes.  A  general  assessment  of  the  lack  of  full 
implementation  of  those  recommendations  was  included  as  Appendix  B  in  our  Phase  I 
Report,  IDA  Document  D-3189.  The  following  two  attributes  appear  to  the  Team  to 
warrant  near-term  Under  Secretary  attention  to  help  improve  accountability  in  the 
Department’s  acquisition  process.  1 

A.  CLEAR  COMMAND  CHANNELS 

Clear  command  channels  are  what  Packard  hoped  to  achieve  by  limiting 
intervening  layers  between  the  program  manager  (PM)  and  the  Acquisition  decision 
maker  (Defense  or  Service  Acquisition  Executive — DAE  or  SAE)  to  no  more  than  two. 
In  Packard’s  view,  the  shortened  decision-making  chain  would  alter  the  nature  of 
oversight  in  a  way  that  would  reduce  the  number  of  staff  overseers  and  speed  up  decision 
making. 
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Figure  IV-1.  Comparison  of  Packard  with  Current  System 
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1  Accountability  is  the  “Liability  to  give  account  of,  and  answer  for,  discharge  of  duties  or  conduct; 
responsibility”  (Oxford  Dictionary) 
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While  DoD  policy  documents  say  the  Department  has  implemented  Packard,  one 
has  simply  to  overlay  a  graphic  of  the  proposed  Packard  chain  of  command  with  the 
Integrated  Product  Team  (IPT)  structure  currently  mandated  by  DoDI  5000.2,  to  see  the 
problem: 

•  First  there  is  the  Overarching  IPT  (OIPT). 

•  Next  there  is  the  mandatory  Integrating  IPT  to  coordinate  the  activities  of 
whatever  subordinate  IPTs  are  deemed  necessary  by  the  OIPT. 

•  Finally,  there  are  the  subordinate  IPTs  called  Working  IPTs.  These  virtually 
always  -  and  appropriately  -  include  a  Cost  IPT  and  a  T&E  IPT 

Clearly,  short  lines  of  communication  have  not  been  established  and  this  failure  is 
a  matter  of  official  policy.  The  OIPT  is  widely  seen  as  a  particular  impediment  to  the 
rapid  and  effective  decision  making  contemplated  by  Packard.  By  increasing  the 
accountability  of  the  subordinate  acquisition  executives,  the  need  for  this  extra  layer  of 
virtual  management  could  be  reduced.  However,  because  cost  estimating  and  test 
planning  span  a  broad  range  of  responsibilities  and  the  entire  program  time  horizon,  these 
two  IPT  activities  should  be  continued. 

B.  CLEAR  RESPONSIBILITY  AND  CONSEQUENCES  FOR  ACTIONS 

Packard  stated  that  “authority  for  executing  acquisition  programs — and 
accountability  for  their  results — has  become  vastly  diluted.” 

We  conclude  that  not  only  has  this  problem  not  been  fixed  in  the  19  years  since 
Packard,  it  may  have  gotten  worse  in  some  ways  despite  implementation  of  Packard’s 
recommendation  to  establish  a  Defense  Acquisition  Executive. 

The  need  for  improved  accountability  is  obvious  in  Packard’s  recommendations 
on  short  decision  chains  and  personal  commitments  for  contracts  between  PMs  and 
Acquisition  executives. 

Paul  Beach,  in  his  1990  report  on  how  the  Navy’s  A-12  stealth  aircraft 
development  program  reached  the  point  where  it  was  canceled  for  cause,2  itself  a  rare 
instance  of  accountability,  described  the  issue  perfectly: 


2 


The  continuing  litigation  costs  of  this  disputed  cancellation  have  now  reportedly  exceeded  the  amount 
in  dispute. 
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The  fundamental  problem  ...  is  to  create  appropriate  incentives  to  enable  senior 
leaders  to  rely  upon  responsible,  accountable  line  managers  for  realistic 
perspectives  on  the  cost,  schedule  and  technical  status  of  their  programs.  Only 
by  doing  so  can  we  increase  efficiency  and  accountability  while  reducing  the 
burdens  imposed  by  undue  regulation  and  stifling  supervision. 

Unless  means  can  be  found  to  solve  this  abiding  cultural  problem ,  the  failures 
evidenced  in  this  report  can  be  anticipated  to  occur  again  in  the  same  or  similar 
form.  ^ 

Policies,  goals,  practices,  and  other  forms  of  enterprise  expectations,  while  they 
are  necessary,  will  not  by  themselves  instill  or  sustain  a  desired  level  of  performance  or 
behavior.  Only  the  nature  and  likelihood  of  consequences — positive  and  negative — can 
do  that,  and  too  often  consequence  is  missing  from  the  DoD  acquisition  environment. 
The  Department  promulgates  more  and  more  policy  directives  to  obtain  the  wanted 
behavior,  but  the  lack  of  consequences  renders  the  directives  ineffective.  When  the 
workforce,  including  middle  management,  perceives  there  are  limited,  if  any, 
consequences  for  following  or  not  following  policies,  that  lack  of  consequence  tends  to 
subordinate  those  policies  to  other  more  immediately  consequential  considerations.  Such 
considerations  can  include  expected  rewards  for  avoiding  taking  management  actions  that 
would  imply  schedule  slips,  or  that  would  trade  minor  perfonnance  goals  favored  by 
senior  service  executives  against  cost. 

If  accountability  is  to  be  improved,  all  levels  and  elements  of  the  acquisition 
community  need  to  know  three  things: 

1 .  That  they  will  be  called  to  account 

2.  What  they  will  be  held  accountable  for 

3.  That  there  will  be  consequences  for  their  actions — or  inactions 

One  of  this  study’s  authors  had  a  discussion  with  a  Service  Acquisition  Executive 
(SAE)  that  went  as  follows: 

In  recurrent  conversations  with  your  PMs,  I  ask,  "What  does  the  SAE  expect 
from  you?”  Invariably,  the  answer  is  “to  bring  my  program  in  on  time,  within 
cost,  and  meeting  performance  goals.”  We  all  know  the  DAU  answer,  so  the 
question  is  rephrased  and  personalized  to,  “What  does  SAE  Smith  expect  from 
you,  Captain  Joe  Jones,  for  your  program,  in  its  unique  state  of  affairs  which  is 
different  from  the  state  of  any  other  program?”  The  answer  to  that  question  is 


3  Chester  Paul  Beach,  Jr.,  “A- 12  Administrative  Inquiry,”  28  November  1990. 
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usually  a  blank  stare,  and  that  is  the  issue.  People  cannot  be  held  to  account  if 
they  do  not  know  specifically  what  is  expected.  The  question  is  rephrased,  “What 
does  SAE  Jones  expect  from  you,  Colonel/Captain/Mr.  Joe  Smith,  on  your 
program,  in  its  current  state  of  affairs  which  is  different  from  the  state  of  any 
other  program?”  The  answer  to  that  question  is  always  a  blank  stare,  and  that  is  a 
major  issue. 

This  disconnect  between  the  formal  acquisition  reporting  chain  and  true 
accountability  exists  largely  because  the  Acquisition  Executives  are  not  the  primary 
conduit  for  long-  or  short-term  program  funding.  This  is  in  direct  contrast  to  industry 
practice  where  a  program  manager’s  agreement  with  his  leadership  usually  includes 
funding  guarantees.  Other  Service  and  DoD  offices  with  authority  over  program  funding 
streams  have  a  major  impact  on  the  DoD  acquisition  Program  Manager’s  ability  to 
execute  the  “contract”  he  has  established  with  the  relevant  Acquisition  Executives. 

The  Annual  Operating  Plan  concept  (AOP)  mitigates  this  persistent  accountability 
problem.  PMs  justifiably  say,  “You  can’t  hold  me  accountable  for  things  I  don’t  control, 
and  the  PPBE  and  appropriation  processes  are  constantly  changing  my  program.” 

If  the  AOP  is  based  on  the  fiscal  year,  most  funding  changes  and  impacts  are  known  by 
the  August/September  time  frame,  which  provides  sufficient  time  to  set  the  next  year’s 
goals  in  a  manner  that  is  almost  entirely  under  the  control  of  the  CAE,  PEO,  and  PM.” 

C.  INITIATIVES  FOR  IMPROVED  ACCOUNTABILITY 

1.  An  Annual  Operating  Plan 

DoD  needs  a  fresh  approach  to  acquisition  accountability,  based  on  the  following 
DoD  Business  Management  Modernization  Program  (BMMP)  realignment  principles: 

1 .  DoD  business  enterprise  clarity; 

2.  Tiered  accountability,  and 

3.  Program  management  discipline. 

Such  an  accountable  management  system  could  best  be  based  on  a  model  that  is 
centered  on  an  annual  operating  plan.  This  would  largely  avoid  the  problem  noted  above 
in  which  non-acquisition  officials  have  authority  over  longer  term  funding. 

An  annual  operating  plan  (AOP)  is  a  schedule  of  events  and  responsibilities  that 
details  the  actions  to  be  taken  in  order  to  accomplish  the  goals  and  objectives  laid  out  in 
the  approved  plan.  The  AOP  ensures  everyone  knows  what  needs  to  get  done, 
coordinates  efforts  when  getting  it  done,  and  keeps  track  of  whether  and  how  it  got  done. 
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In  the  current  DoD  acquisition  environment,  the  Acquisition  Program  Baseline 
(APB)  serves  as  the  long-term  plan.  The  trouble  with  the  APB,  and  long-term  plans  in 
general,  is  that  they  commonly  extend  so  far  into  the  future  that  it  is  impractical  to  hold 
anyone  accountable  for  their  eventual  achievement.  For  example,  how  can  a  Component 
Acquisition  Executive  (CAE),  Program  Executive  Officer  (PEO),  or  Program  Manager 
(PM)  be  held  accountable  for  achieving  Initial  Operating  Capability  (IOC)  when  it  is  10 
to  15  years  or  more  in  the  future?  Obviously  they  cannot.  But,  an  AOP  based  on 
approved  1-year  funding  could  establish  and  measure  progress  on  those  events  in  the  next 
12  months  that  are  on  or  near  the  critical  path  to  IOC.  Progress  over  the  upcoming  year 
can  easily  be  measured  and  appropriate  persons  can  be  held  accountable  for  progress. 
Failure  to  achieve  the  annual  goals  would  be  an  early  indicator  of  a  potential  slip  in  IOC. 
If  the  AOP  is  based  on  the  fiscal  year,  most  funding  changes  and  impacts  are  known  by 
the  August/September  time  frame — early  enough  to  allow  the  next  years’  goals  to  be  set 
in  a  manner  that  is  almost  entirely  under  the  control  of  the  CAE,  PEO,  and  PM. 

The  establishment  of  an  Annual  Operating  Plan  would  more  properly  align 
responsibilities  and  authority  within  the  Department’  acquisition  process,  and  would 
pennit  CAEs,  PEOs,  and  PMS  to  become  more  objectively  accountable  for  the  progress 
of  their  programs. 

Additional  detail  on  establishing  and  utilizing  DoD  Acquisition  Annual  Operating 
Plans  is  presented  in  Appendix  B. 

2.  A  More  Flexible  and  Accountable  Acquisition  Workforce 

The  AOP  is  only  one  aspect  of  the  accountability  issue.  The  AOP  would  enhance 
organizational  clarity  and  establish  the  expectations  upon  which  accountability  can  be 
based,  but  accountability  requires  at  least  two  participants — the  one  being  held 
accountable,  and  the  one  doing  the  accounting.  The  first  is  easily  established,  but  finding 
leaders  who  will  do  the  accounting  is  another  matter.  Instilling  accountability  and 
achieving  “fundamental  change  in  the  department’s  acquisition  process”  depends  on 
leadership.  The  next  step,  then,  is  to  take  control  of  the  acquisition  workforce  to  nurture 
and  reinforce  an  environment  of  accountability. 

DoD  needs  an  acquisition  workforce  that  is  well  trained,  broadly  experienced,  and 
mobile.  The  Defense  Acquisition  Workforce  Improvement  Act  (DAWIA)  establishes  a 
reasonable  starting  point,  but  more  improvements  are  needed.  In  particular,  management 
must  be  able  to  unilaterally  deploy  personnel  where  their  talents  and  backgrounds  are 

III-5 


most  needed.  In  addition,  management  needs  to  be  actively  engaged  in  career  planning  to 
ensure  a  deep  pool  of  the  best  talent  available.  Actively  engaged  means  people  need  to 
be  told  when  they  are  falling  short  and  when  they  need  career  broadening  experience. 
Such  management  should  adopt  for  the  civilian  acquisition  workforce  some  of  the 
characteristics  of  the  Service’s  officer  management  programs.  That  does  not  have  to 
mean  an  up-or-out  policy,  but  it  may  mean  an  up-or-stav  put  policy.  Movement  into  any 
Critical  Acquisition  Position  (CAP),  or  promotion  from  one  CAP  to  a  more  senior 
position,  should  require  increasingly  rigorous  qualifications  for  breadth  of  experience  and 
depth  of  training.  Management  should  facilitate  workforce  mobility  at  the  local,  Service, 
and  Agency  level  to  enhance  the  preparation  of  those  acquisition  workforce  members 
who  demonstrate  the  desire  and  ability  to  be  acquisition  leaders. 

On  the  other  hand,  if  the  current  practice  of  rotating  military  program  managers 
too  frequently  cannot  be  seriously  ameliorated,  serious  consideration  should  be  given  to 
staffing  most  DoD  program  offices  with  civilian  Program  Managers  who  will  be  expected 
to  stay  in  place  until  the  program  succeeds  (or  is  canceled)4.  Military  officers  could  bring 
their  valuable  battlefield  experience  to  bear  as  rotating  deputy  program  managers. 

Succession  planning  should  be  established  for  ACAT  I  and  II  PMs,  PEOs,  and 
higher-level  CAPs,  to  mitigate  the  adverse  impact  of  routine  management  rotations  and 
other  changes.  Properly  done,  this  will  enable  replacement  by  people  who  understand  the 
nature  of  both  their  programs  and  the  policy  initiatives  of  the  current  senior  leadership. 

Other  enablers  of  a  high-quality  workforce  include: 

•  Strict  limitations  on  the  amount  of  non-acquisition  experience  (e.g., 
“command”  time)  pennitted  for  qualification. 

•  Elimination  of  the  grace  period  that  allows  individuals  up  to  18  months  to  get 
qualified  in  a  position  to  which  they  are  already  assigned — no  more  OJT  that 
lasts  for  up  to  half  the  PM’s  tenure. 

•  Require  all  PM  and  higher  level  CAPs  to  be  filled  with  persons  Level  III 
certified  in  at  least  one  DAWIA  specialty  other  than  that  required  for  the 
particular  CAP. 

•  Require  and  enforce  mobility  agreements  for  all  civilian  CAPs. 


4  Most  Naval  officers  that  led  the  development  of  the  Navy  Tactical  Data  System  (NTDS)  development 
were  with  that  successful  project  for  at  least  six  years  -  more  than  twice  the  current  practice.  When 
Computers  went  to  Sea;  David  L  Boslaugh,  1999. 
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•  Require  senior-level  CAPs  to  have  experience  with  multiple  acquisition 
organizations  with  cross-Service  and  Agency  experience  preferred. 

D.  RECOMMENDATIONS  FOR  IMPROVING  ACCOUNTABILITY  IN  THE 
ACQUISITION  WORKFORCE 

1.  The  Under  Secretary  of  Defense  (AT&L)  should  establish  a  new  oversight  process  that 
utilizes  tiered  accountability. 

•  An  Annual  Operating  Plan  with  goals  and  objectives  should  supplement  the 
APB  as  the  primary  agreement  for  the  next  12  months  for  which  each  level  of 
management  will  be  held  accountable. 

•  Quarterly  reviews  should  be  restructured. 

o  Hold  on-site  reviews  in  the  facilities  of  each  host  Component  (as  a 
practical  matter,  non-Service  Components  could  be  at  a  mutually 
acceptable  location). 

o  Hosting  Component  sets  the  agenda  based  on  the  progress  against  the 
AOP  (this  should  not  preclude  the  USD(AT&L)  from  specifying  a  limited 
number  of  special  interest  items). 

•  The  annual  operating  plan  should  be  structured  and  implemented  in  a  manner 
similar  to  a  balanced  scorecard.5 

2.  The  Under  Secretary  of  Defense  (AT&L)  should  cut  back  on  the  size  and  function  of 
the  OIPT  structure. 

•  The  Component  Acquisition  Executives  should  be  held  responsible  and 
accountable  for  their  programs.  They  should  determine  when  their  programs  are 
ready  for  a  milestone  review  within  the  guidelines  of  DoD  acquisition  policy 
(including  the  systems  engineering  guidelines  discussed  in  the  Section  II  of  this 
report)  and  they  should  have  the  authority  to  schedule  such  reviews  with  the 
Defense  Acquisition  Executive  when  they  believe  they  are  ready. 

•  DAE  staff  should  be  responsible  for,  and  given  enough  lead  time  and  information 
by  the  CAE)  for  preparing  the  DAE  for  the  review  but  not  for  deciding  when  and 
whether  to  hold  a  review.  Accountability  requires  the  review  to  be  held  when  the 
responsible  CAE  claims  readiness.  If  it  turns  out  the  program  is  not  ready,  the 
CAE  is  sent  back  with  an  admonition  not  to  call  reviews  for  which  they  are  not 
prepared. 


5  For  a  detailed  description,  see  Gary  E.  Christie  et  at,  “Improving  Acquisition  Metrics,  Phase  II,” 
Center  for  Naval  Analyses,  October  2002. 
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3.  The  Under  Secretary  of  Defense  (AT&L)  should  use  the  flexibdity  inherent  in  the  new 
National  Security  Personnel  System  to  exercise  aggressive  leadership  in  career 
management,  creating  a  truly  professional  acquisition  corps  that  is  well  and  broadly 
trained,  and  easily  deployed  on  an  individual  basis  to  where  the  skills  are  needed. 

•  Strictly  enforce  qualification  requirements  for  PM  selection. 

•  Require  all  PM  and  higher  level  CAPs  to  be  filled  with  persons  Level  III  certified 
in  at  least  one  DAWIA  specialty  other  than  that  required  for  the  particular  CAP. 

•  Implement  succession  planning  for  PM  and  higher  CAPs. 

•  Ensure  the  final  version  of  the  National  Security  Personnel  System  (NSPS)  will 
support  the  above  actions. 
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IV.  INDUSTRIAL  BASE  AND  ACCESS  TO 
TECHNOLOGY  ISSUES 


As  noted  in  the  Phase  I  report,  the  Team  examined  several  acquisition  reform 
initiatives  that  are  related  to  industrial  base  and  access  to  technology  issues.  We  found 
that,  although  many  such  specific  issues  rise  to  the  attention  of  the  Under  Secretary  in  the 
nonnal  course  of  the  Department’s  activities,  there  are  no  management  issues  in  this 
sector  that  are  sufficiently  urgent  to  warrant  a  major  new  USD(AT&L)  initiative.  The 
following  material  summarizes  our  evaluation.  The  first  topic  identifies  a  potentially 
useful  minor  initiative. 

A.  CHALLENGES  TO  DOD  ACCESS  TO  TECHNOLOGY 

The  day  has  long  passed  when  the  Defense  Department  could  rely  exclusively,  or 
even  primarily,  on  technology  that  had  been  developed  as  a  result  of  DoD  investments. 
The  explosion  of  new  applied  technology  in  the  US  commercial  sector,  coincident  with 
globalization  of  such  developments,  poses  a  significant  challenge  to  DoD.  No  longer  can 
government  laboratories  and  traditional  defense  contractors  be  looked  to  as  the  primary 
source  of  new  technologies  important  to  future  defense  systems.  Indeed, 
acknowledgement  of  this  trend  lies  behind  many  of  the  acquisition  “reforms”  adopted  by 
the  Department  over  the  past  decade,  including  its  enthusiasm  for  the  use  of  Other 
Transaction  Authority  (OTA)  agreements  to  hopefully  gain  greater  access  to  non- 
traditional  suppliers. 

But  applied  military  technology  flows  out  of  basic  research,  which  in  this  country, 
with  the  exception  of  drugs,  is  still  largely  dependent  on  federal  funding.  The  problem 
for  DoD  is  being  exacerbated  both  by  the  ongoing  decline  in  the  federal  investment  in 
basic  and  applied  research,  and  by  the  Defense  Department’s  decision  in  the  1990s  to 
cease  giving  industry  incentives  to  spend  its  government-reimbursed  independent 
research  and  development  funding  on  long-tenn  science  and  technology  projects  that  are 
of  particular  importance  to  national  defense  needs. 
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B.  BROAD-BASED  SOLUTIONS  FOR  IMPROVING  DOD  ACCESS  TO 
TECHNOLOGY 

The  simple  solutions  to  the  aforementioned  challenges  would  be  to — 

1 .  Reverse  the  decline  of  federal  spending  on  basic  research  by  increasing  DoD 
and  other  federal  agency  investments  in  this  area. 

2.  Resume  the  former  DoD  practice  of  “scoring”  industry  independent  research 
and  development  (IR&D)  projects  against  the  Department’s  long-term  goals 
when  detennining  the  level  at  which  such  investments  would  be  reimbursed 
via  DoD  contract  overhead  allowances.  Such  an  action  is  well  within  the 
purview  of  the  USD(AT&L)  and  is  included  here  as  a  potential  new  initiative 
worth  pursuing. 

The  National  Academy’s  Committee  on  Prospering  in  the  Global  Economy  of  the 
21st  Century  recently  produced  a  study  for  the  Senate  Energy  and  Natural  Resources 
Committee.  This  study  emphasized  that  the  federal  government  is  the  only  source  for 
funding  basic  research  in  the  United  States;  that  corporate  R&D  funding  is  product- 
oriented;  and  that  many  of  today’s  most  successful  commercial  technologies  originated  in 
basic  research  funded  by  the  Department  of  Defense,  where  support  for  such  funding 
continues  to  wane.  The  study  recommended  that  DoD  funding  of  basic  research  be 
increased  at  a  rate  of  10%  per  year.  Such  a  funding  increase  would  also  benefit  efforts  to 
increase  the  availability  of  the  meaningful,  interesting,  and  important  research  work 
needed  to  further  motivate  US  students  to  pursue  challenging  technical  and  scientific 
education  goals. 

The  more  complex  issue  involves  DoD  access  to  advanced  technologies  whose 
centers  of  excellence  are  outside  the  United  States.  To  date,  DoD  has  relied  primarily  on 
its  large,  multinational  prime  contractors  to  manage  such  access,  and  this  may  continue  to 
be  the  best  approach.  We  are  making  no  specific  recommendations  for  a  new  initiative  in 
this  area,  which  the  Team  does  believe  warrants  increased  attention. 

C.  INDUSTRIAL  BASE  ISSUES 

Just  as  the  American  public  broadly  benefits  from  the  growing  globalization  of 
the  consumer  economy,  within  limits  the  Defense  Department  also  broadly  benefits  from 
the  globalization  of  the  supply  chain  both  for  the  lowered  cost  of  its  commercial  product 
needs  and  for  its  access  to  advanced  technologies  for  which  the  US  is  not  a  leader.  But 
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for  supplies  for  which  a  surge  capacity  is  assessed  as  an  important  element  of  US  national 
security  planning,  there  is  no  reason  to  depart  from  the  current  practice  of  funding  such 
standby  capacity  in  the  US.  Such  needs  include,  for  example,  vaccine  production, 
antidote  production,  other  limited  shelf-life  supplies,  and  some  types  of  ammunition. 

It  is  also  important  that  the  equipment  on  which  the  United  States  relies  for  its 
most  sensitive  communications  and  intelligence  activities  are  assembled  from  “trusted 
sources”  of  components.  This  area  was  also  the  subject  of  a  recent  report  by  the  National 
Academy,1  which  recommended  that  DoD  maintain  its  current  “in-house”  abilities  to 
produce  trusted  printed  circuit  boards,  while  taking  specific  steps  to  assure  such  secure 
capabilities  for  the  future. 

There  are  obviously  other  strong  incentives  for  the  United  States  to  ensure  that  its 
industrial  base  can  continue  to  produce  the  principal  weapons  systems  that  are  used  to 
equip  its  military  forces.  However,  as  DoD  becomes  increasingly  dependent  on 
technology  for  which  other  free  world  countries  may  have  gained  a  competitive 
advantage,  it  is  unrealistic — even  counterproductive — to  demand  that  arbitrary 
percentages  of  DoD  equipment  components  and  software  originate  in  the  United  States. 
Furthermore,  paying  for  the  maintenance  of  excess  defense  industrial  production  capacity 
in  the  hopes  of  reducing  costs  though  competition  is  also  generally  counterproductive,  as 
discussed  below. 

Paying  for  extra  capability  to  design  and  prototype  new,  innovative  forms  of 
military  equipment  may  well  be  worthwhile,  but,  as  also  discussed  below,  such  a  program 
would  need  to  be  made  profitable  in  its  own  right  to  be  successful. 

D.  COMPETITION 

There  are  two  chief  perceived  benefits  of  formal  competition  in  defense 
acquisition  programs:  design  innovation  and  cost  reduction. 

1.  Design  Innovation 

Competition  is  very  effective  in  bringing  forth  the  best  industry  can  offer  at  the 
beginning  of  every  major  new  acquisition  program.  Top  talent  is  frequently  switched 


1  Linkages:  Manufacturing  Trends  in  Electronics  Interconnection  Technology,  National  Academy  Press, 
2005. 
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from  lucrative  ongoing  programs  to  help  fonnulate  the  company’s  technical  concepts  for 
the  big  competition  at  hand.  And  the  reason  for  this  success  is  not  hard  to  discern;  the 
companies  know  that  the  winner  probably  will  never  have  to  face  further  real  competition 
on  that  program.  For  this  reason,  bidders  not  only  commit  their  best  design  talent,  but 
also  frequently  promise  to  share  the  cost  of  the  early  development  phase  of  the  program. 
Some  may  still  believe  this  is  a  good  deal  for  the  government;  the  team  does  not.  By 
accepting  such  in-kind  “contributions”  early  in  development,  the  government  sub¬ 
optimizes  its  long-term  interests  and  makes  some  implicit  commitment  that  it  will 
proceed  into  full-scale  development  and  production.  Such  a  commitment,  whether 
implicit  or  not,  limits  the  government’s  ability  to  decide  on  alternate  courses  of  action. 

Furthermore,  if  the  government  places  any  significant  weight  on  such  “up-front” 
contributions  when  selecting  the  prime  contractor,  it  may  well  forgo  much  larger  benefits 
available  from  other  bidders  in  tenns  of  lower  future  production  and  operating  cost 
and/or  better  system  perfonnance  features.  Such  considerations  have  motivated  the 
Department’s  growing  use  of  “best  value”  source  selection  criteria  in  recent  years. 

At  least  the  early  phases  of  research  and  development  (R&D)  activities  should  be 
made  profitable  in  their  own  right,  without  the  promise  of  a  production  run  to  “get  well.” 
Such  an  approach  could  greatly  increase  the  government’s  ability  to  keep  competent 
design  teams  productively  employed  without  the  obligation  to  take  designs  to  production 
before  they  may  be  needed.  It  might  also  bring  into  the  DoD  orbit  many  nontraditional 
R&D  firms  that  may  be  able  to  contribute  innovative  ideas.  But  this  would  be  a  hard  sell, 
in  part  because  of  the  very  real  intellectual  property  ownership  issues  that  surround  such 
programs.  As  discussed  earlier  in  this  report,  the  pre -milestone  B  risk-reduction  process 
could  be  strengthened  by  adequate  and  profitable  funding  of  technology  demonstration 
and  validation  of  alternative  technologies  prior  to  formal  decisions  on  the  structuring  of  a 
major  program. 

There  are  also  always  ongoing  pressures  to  only  invest  significantly  in 
developments  for  which  there  is  a  follow-on  production  program.  The  Joint  Strike  Fighter 
program  started  as  a  series  of  design  and  prototype  testing  competitions  for  advanced 
aircraft  components  and  subsystems,  but  industry,  and  perhaps  the  Congress,  quickly 
insisted  that  such  expenditures  would  be  justified  only  if  an  aircraft  development  program 
was  established  in  the  funded  program  of  record.  It  will  take  sustained  senior  leadership 
to  ensure  that  risk-reduction  hedges  do  not  gain  enough  political  momentum  to  become 
unwarranted  duplicative  acquisition  programs. 
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2.  Cost  Reduction 


In  contrast  to  the  benefits  of  formal  design  competition,  the  cost  reduction 
benefits  of  competition  are  often  overestimated,  at  least  at  the  major  system  level.  Indeed 
once  a  major  defense  contractor  has  won  a  design  competition  for  a  complex  major 
system  such  as  a  new  combat  aircraft,  or  class  of  ship,  and  any  subsequent  down-select 
that  is  intended  to  lead  to  production,  the  threat  of  further  competition  will  have  largely 
vanished  At  that  point  the  contractor’s  duty  to  his  shareholders  to  keep  costs  and  profits 
up  on  the  prevalent  cost-type  contracts  begins  to  conflict  directly  with  the  government’s 
interest  in  driving  costs  down.  The  only  exception  is  the  currently  infrequent  situation  in 
which  the  production  rates  are  expected  to  be  high  enough  that  the  cost  of  funding  a 
second  source  would  likely  be  more  than  offset  by  the  savings  from  competitive  pressure 

Once  a  qualified  prime  contractor  is  producing  satisfactory  equipment  under  a 
prime  contract,  the  government  usually  needs  to  employ  tools  other  than  direct  re¬ 
competition  to  encourage  cost  limitations  and  reductions.  These  tools  take  many  forms, 
such  as  detailed  tracking  of  the  contractor’s  actual  costs,  directed  component  break  out, 
and  award  and  incentive  fees,  and  are  highly  unique  to  government  management 
practices.  The  private  sector  has  very  few,  if  any,  long-term  cost-plus  contractual 
relationships  where  the  buyer  has  no  alternate  supplier  reasonably  available. 

3.  Other  Types  of  Competition 

There  is  a  third  potential  use  of  competition  in  defense  acquisition  that  has  not 
been  generally  adopted  but  may  be  worth  additional  attention.  This  is  the  notion  of  cross¬ 
system  and  even  cross-Service  competition  for  funding  to  meet  a  real  “mission”  or 
“capability”  need.  Such  an  approach  has  been  suggested  by  past  Defense  Science  Board 
task  forces  through  such  broad  examples  as  comparing  the  costs  of  striking  inland  targets 
from  Navy  carriers  with  the  costs  of  Air  Force  bombers  for  the  same  effects.  The 
Department  is  not  currently  organized  or  staffed  to  routinely  conduct  such  studies  of 
broad  alternatives  “in  house.”  Having  conducted  a  Deep  Attack  Weapons  Study  along 
these  lines  for  OSD  several  years  ago,  IDA  can  attest  to  the  difficulties  of  such  attempts 
at  explicit  cross-service  competition. 

The  Department  at  one  time  championed  a  “Challenge”  program  in  which  outside 
suppliers  could  formally  offer  to  provide  some  piece  of  equipment,  or  subsystem,  to  DoD 
at  a  lower  price  than  was  currently  on  contract.  The  opportunities  for  benefiting  from 
such  a  program  have  probably  declined  in  recent  years  as  the  Department  has  undertaken 
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less  and  less  of  its  own  system  integration  work,  thereby  reducing  its  ability  to  switch 
sources  for  components  or  subsystems.  Nevertheless,  some  such  new  effort  to  open 
ongoing  DoD  contracts  to  new  ideas  and  technologies  from  outside  suppliers  may  well  be 
warranted. 
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Appendix  B 

THE  ANNUAL  OPERATING  PLAN  CONCEPT 


Implementing  an  annual  operating  plan  (AOP)  requires  that  goals  and  objectives 
be  set  and  regularly  monitored.  Most  companies  do  this  with  something  that  resembles  a 
Balanced  Scorecard  (BSC). 


The  generic  BSC  integrates  an  organization’s  vision  and  strategy  and  typically 
has  four  elements:  Financial,  Processes,  Customers,  and  People. 

In  turn,  each  element  has  objectives,  measures,  targets,  and  initiatives. 

In  a  commercial  enterprise,  “Financial”  is  usually  the  most  important  element 
because  enhancing  shareholder  value  is  usually  the  enterprise’s  reason  for  existence.  The 
other  three  elements  are  the  means  by  which  the  financial  objectives  are  achieved,  and 
the  purpose  of  the  BSC  is  to  ensure  the  other  elements  are  not  sub-optimized. 


For  the  DoD  acquisition  enterprise,  the  BSC  should  be  adopted  to  reflect  the 
reason  for  “Acquisition,”  i.e.,  the  delivery  of  systems  and  capabilities  to  the  warfighter 
customer.  Financial  metrics  are  usually  important  to  DoD,  but  they  are  measures  that 
should  be  incorporated  under  each  element  as  appropriate.  Figure  B-l  shows  one  way  to 
adapt  the  traditional  BSC  to  DoD  Acquisition. 


How  do  our  customers  see  us? 


Do  we  provide 
the  best  capability* 
for  the  warfighter? 


Do  we  continue  to 
Improve  and  create  value? 


Figure  B-1.  DoD  Acquisition  Balanced  Scorecard 

The  BSC  is  the  framework  within  which  the  goals  and  objectives  of  the  AOP 
should  be  established,  along  with  metrics  to  measure  progress  throughout  the  year. 
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Secretary  Krieg  has  raised  a  concern  with  metrics,  as  follows:  “We  measure 
everything,  but  by  measuring  everything  and  aligning  nothing  at  senior  levels,  we  really 
measure  nothing.”1  The  AOP/BSC  addresses  this  concern  by  establishing  a  means  to 
continually  measure  progress  towards  a  limited  number  of  goals  and  objectives  that  are 
tailored  to  each  Component  Acquisition  Executive  (CAE),  Program  Executive  Officer 
(PEO),  and  Program  Manager  (PM. 

Figure  B-2  shows  one  way  to  flow  down  the  AOP/BSC.  Note  that  the  flow  goes 
down  to  the  staff  level  as  well  as  to  the  AT&L  Principal  Staff  Assistants.  In  this  way  all 
participants,  at  all  levels,  have  a  stake  in  achieving  common  outcomes.  At  appropriate 
levels  this  flow  of  the  AOP/BSC  supports  the  new  National  Security  Personnel  System 
performance  requirements  for  civilians  and  should  similarly  support  unifonned  fitness 
reports  as  well. 


Staff 


Figure  B-2.  Balanced  Scorecard  Flow  Diagram 

The  AOP/BSC  should  be  combined  with  a  new  oversight  process  that  replaces  the 
current  Defense  Acquisition  Executive  Summary  (DAES)  process,  wherein  CAEs  are 
called  to  respond  to  an  agenda  largely  established  by  the  AT&L  staff,  with  a  process  that 
recognizes  who  is  responsible  and  accountable  for  execution,  and  reflects  how  those 
responsibilities  are  tiered.  The  review  agendas  should  be  generally  set  by  the  host  CAE 
and  should  be  focused  primarily  on  execution  of  the  AOP.  This  focus  should  include  an 


1  Address  to  DAU  faculty  and  students  on  transforming  the  processes  and  decision  tools  in  the 
Department  of  Defense  (DoD),  as  reported  in  Defense  AT&L,  January-February  2004. 
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aggregate  portfolio  view  with  individual  programs  addressed  on  an  exception  basis.  In 
the  private  sector,  the  AOP/BSC  is  usually  combined  with  quarterly  on-site  reviews  of 
business  unit  portfolios,  supplemented  by  monthly  reporting,  usually  of  financials. 
Earned  value  reporting  could  be  a  useful  surrogate  for  the  industry  financial  reporting 
process.  The  data  is  produced  by  the  contractor  in  the  routine  execution  of  significant 
contracts,  is  an  objective  measure  of  progress,  and  imposes  virtually  no  reporting  burden 
on  the  PMO. 
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14.  ABSTRACT 

Our  review  focused  on  the  processes  and  procedures  for  the  long-term  acquisition  of  military  equipment  and  systems  essential  for  properly 
equipping  and  supporting  the  future  military  force  structure.  In  the  Phase  II  report  the  team  proposes  three  major  sets  of  initiatives  and  one 
minor  initiative  for  further  action  by  the  USD(AT&L). 


15.  SUBJECT  TERMS 


acquisition  reform,  acquisition  excellence,  acquisition  initiatives 


17.  LIMITATION 

18.  NO.  OF  PAGES 

19a. NAME  OF  RESPONSIBLE  PERSON 

OF 

16.  SECURITY  CLASSIFICATION  OF: 

ABSTRACT 

60 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE 

uu 

19b.  TELEPHONE  NUMBER  (Include  Area 
Code) 

u  u  u 

Standard  Form  298  Back  (Rev.  8/98) 


